Design of an Embedded Processor for Smart Camera Systems
Project Overview

Smart cameras represent a quantum leap in sophistication over commonly available digital cameras. Smart Cameras capture high-level description of a scene and perform real-time analysis of what they see. Moving well beyond pixel processing and compression, these systems run a wide range of algorithms to extract meaning from streaming video. These devices can support a wide variety of applications including surveillance, motion analysis, traffic monitoring, etc. Developments in this area have, so far, primarily focused on the development of passive smart camera systems.. In this project we propose to develop ASIPs which will facilitate implementation of active smart cameras. These ASIPS will have enriched instruction set to facilitate intelligent processing of captured images and purposive movement of the camera based upon visual feedback.

  • Design and analysis of appropriate real time vision algorithms for identifying desirable hardware features for the target architecture for implementation of smart camera system
  • Design and analysis of a motion control scheme for the camera with visual feedback for identifying desired hardware block in the ASIP
  • Design and development of an ASIP for facilitating implementation of active smart camera system

Smart camera system essentially consists of an image acquisition block followed by a processing module built around general purpose processor and/or a digital signal processor. Key challenge lies in the development of real time algorithms for the processing architecture to extract high level descriptors from the streaming video. In this project, we propose to analyze and develop real time vision algorithms for tracking objects, human action and interpreting activities. The algorithms apart from standard accuracy requirements, will need to satisfy memory, frame rate and latency constraints for the target processing architecture. Latency, time taken for producing results for a frame, is a critical constraint because this output is used in closed loop control system for purposive motion control of the camera. These issues actually necessitate development of special purpose processor architecture different from traditional DSP's. Instruction set architecture, which has a judicious combination of signal processing operations and efficiency of RISC architecture can actually facilitate efficient implementation of these class of algorithms.

  • The design is proposed to be done using high level design tools (VHDL and synthesis tools) in a top-down technology independent manner with possibilities of retargetable physical implementation using any of the physical design methodologies and scalable technology.
  • It is proposed to model the processor at various levels of abstraction which include a purely behaviour level model, detailed micro-architectural level model with encoded control store patterns before getting down to the logic level synthesis and implementation based on a specific design library of a foundry.
  • The project involves designing of the ASIP and a prototype electronic system based on it for use in active smart camera.
  • The design will pave the way to the development of design capabilities of image processor through the specific design of this image processor. The project will also serve to generate and integrate technological knowledge base and competence in key technological areas of VLSI architecture design and image processing.

Prof. Shantanu Chaudhary

Kavita, Ritu, Ruchi

Minisitry Of Communication and Information Technology

Participating Institutes
  • IIT-Delhi
  • CEERI Pillani
  • IIT Bombay
  • Jadhopur University
  • IIIT Hyderabad
  • CDAC-Kolkata