Huge amounts of raw imagery collected by sensors such stationary pan-tilt-zoom cameras and mobile platforms (UAV, UGV, satellite, etc.) require more human resources for analysis than are available. As a result, most of the collected imagery is either archived or thrown away without ever being analyzed.
The objective of this research program is to develop theory, efficient computational methods and tools to enable automated analysis of imagery. Applications of such a capability are numerous and include reducing the workload of image analysts, automated surveillance of wide areas over long periods of time with a network of stationary and mobile cameras, rapid detection of potential threats, indexing and archiving images based on their content and concise description of the image/scene. Image understanding is also a critical component of perception in robotics operations for navigation and obstacle avoidance, manipulation of objects and interactions with humans.
Image understanding involves detecting, localizing and recognizing objects, tracking and recognizing activities of moving objects, recognizing the scene type (street scene, commercial area, residential area, hallway, sitting room, etc.), discriminating between background and foreground, determining what is important (which depends on the task) and coupling this information to the automated understanding system with the ultimate goal of inferring intentions, potential threats and opportunities.
Factors that make image understanding a challenging research area include the fact that there are thousands of object classes in the world and there are a very large, but unknown, number of activities. Also, a typical image contains 10s-100s of stationary objects and several to tens of moving objects and the appearance of objects can change with changes in illumination, view point and occlusion. Additionally, activities may appear different because of their inherent temporal variations.
Current emphasis of this program is on understanding EO/IR image/video. The longer term goal is to extend these methods to other modalities of imaging.