Video IoT


  • Bharadwaj Amrutur (RBCCPS/Electrical Communication Engineering)
  • Raghu Krishnapuram (RBCCPS)
  • Abhay Sharma (RBCCPS)

Cameras are among the key data resources for Smart Cities. They have the potential to enable smart applications that touch almost all aspects of daily urban life: Automated surveillance, intelligent transport management systems, traffic management systems, smart parking systems, and infrastructure monitoring to name a few.

To enable multiple smart city applications by multiple solution providers, it is critical that Smart Cities adopt an API based, platform approach towards exchanging and consuming video and media data, similar to those for IoT data.

We are currently working to create such a platform. In particular, we have the following focus areas:

  1. [Media Meta-Data]: Create a framework to associate an meta-data stream to raw media stream, that contain additional information of relevance to the semantic content of the raw media stream, and that enables easy discovery and additional value creation by downstream analytics applications.  
  2. [Media Data Exchange]: Create an  API based, scalable, media data exchange platform, that allows for easy discovery of media resources, secure and easy management of media devices, allows for secure, authenticated, privacy preserving and accountable access for both supplying and consuming live and archived media streams.

One key enabler for successful usage of data is the availability of meta-data, i.e., data about the data. Example meta information for video data include: Time of capture, camera pose, camera location, etc. We are working on creating a standard framework to capture and report the meta-data of media streams like video or audio.

While many encoding formats already capture some meta-information, they are mostly about the encoding aspects. In addition to those, from  the perspective of analytics especially, it is very useful to also have meta-data that is about the semantic content of the streams. These meta-data streams can be generated by video-analytic software or by an agent that has access to other sensors. A key idea is to treat these meta-data streams also as any other IoT sensor stream. That is, these can be consumed via the platform and have an associated data-schema (that describes what information is contained in the stream) and this schema is stored in the resource catalog.

IoT view of video data