Projects

• Home
• Research
  • Human Gesture Analysis
  • Smart Environments
  • Smart Camera Architecture
  • Vision-based Localization
  • Surveillance
• People
• Publications
• Download
• Courses
• Contact

Human Gesture Analysis

• Collaborative Gesture Analysis in Multi-Camera Networks

  An architecture for opportunistic discovery of gesture elements for analysis of human gestures in a multi-camera sensor network is developed in this project. The proposed approach is motivated by the diversity of gestures expressed in passive monitoring applications, and is based on the concept of opportunistic fusion of simple features within a single camera and active collaboration between multiple cameras in the decision making process. This means that within a single camera, a number of simple features are aggregated adaptively for the model, whereas between multi-view cameras, collaboration is pursued in different levels to employ the available pieces of information to provide a description of the gesture. By reducing the uncertainty through different levels of collaboration, the proposed opportunistic approach offers the potential to address gesture recognition problems more efficiently and accurately.

• Real-Time Pose Analysis for Gaming

  a network of smart cameras is used to develop a real-time gaming application based on detecting the user’s posture. We employ a wireless smart camera with a programmable Single-Instruction Multiple-Data (SIMD) processor dedicated for image processing. Another embedded processor also sits inside the smart camera which takes in the output of the SIMD processor for further processing and then sends the results through a ZigBee channel. As the outputs of the processors in the smart cameras, the centroids of the body parts are detected and sent to the central PC to observe the bandwidth constraints of ZigBee. In the central PC, noise is filtered both within single camera’s data and also by combining observations from two cameras. This helps to mitigate detection jitter caused by sensor noise in the cameras. Finally the 3D skeleton model of the user is estimated from the 2D body part centroids. As the application, we place two users into a virtual ball-play game to create interactions between the users and the system. The system operates in real time with a 30 frames/sec rate.

• Collaborative Face Orientation and Profile Analysis

  Most face recognition and tracking techniques employed in surveillance and human-computer interaction (HCI) systems rely on the assumption of a frontal view of the human face. In alternative approaches, knowledge of the orientation angle of the face in captured images can improve the performance of techniques based on non-frontal face views. In this work, we developed a collaborative technique for face analysis in smart camera networks with a dual objective of detecting the camera view closest to a frontal view of the subject, and estimating the face orientation angles in all the camera views based on additional fusion of local angle estimates. Soft information indicating the probabilities of face and eye candidates in each image is exchanged between the cameras, and epipolar geometry mapping is employed to assess correspondence between candidates in different views. Once the camera with the closest view to the frontal face view is identified, further exchange of the face orientation angles estimated by all cameras allows for a collaborative refinement of the estimates according to their associated confidence levels. The develpoped collaborative detection and estimation schemes employ low-complexity algorithms and do not require image transfer between the cameras. Hence, these schemes are applicable to networks of image sensors with in-node processing and narrowband wireless communication.