SentiSight algorithm demo shows main features learning and recognition of SentiSight library. SentiSight Library is intended for visual appearance based object recognition. Thus, its main features are object learning and recognition. The first part, object learning, is dedicated to extraction of object model from a sequence of images containing the object. The second part, recognition, stands for object recognition in a test image. Demo application enables learning and recognition from image file, directory, video file and camera. Image file allows selecting one or few images in one time. Directory allows selecting directory with images. Video file enables to select video file. Camera option allows save avi file for object selection and for recognition enables to live recognition. Camera option is the same in recognition as live recognition. Video files, image lists and images for testing can be downloaded from website: http://www.neurotechnology.com/.
In order to recognize an object in an image, the appearance of the object should be memorized. The process of memorizing an appearance of the object from images with various poses is called object learning. A set of images containing the object should be provided to the algorithm and the algorithm extracts so called model - a symbolic representation of the object. It is highly recommended to provide information about exact location of the object in the image. This can be done by the shapes of the object. Shapes explicitly specifies the object. Thus, only object specific information will be included into model template.
The quality of object recognition highly depends on model created by object learning part. Thus, a set of images of the object should contain all possible poses of the object - the three dimensional rotations (off plane rotations) are highly recommended. Also, it is recommended that images of the object would be taken under different light conditions or using different light sources in order to improve invariance to diverse light conditions. It is recommended to use shapes.
Recognition is a process of identification whether an image contains an object or not. Recognition compares models which were learnt by object learning part, with current test image or test model, and returns a comparative score (score or similarity). A high similarity score suggests that the test image contains one of the learnt objects. On the other hand, a low similarity score implies that the test image contains noise, background or unknown object.
SentiSight Library performs most excellent with constant exterior objects. The algorithm uses texture to localize the object, so highly discriminatively textured objects are recognized better than poorly textured ones. Also, recognition of objects with some moving parts could decrease due to local changes of textures. This is also the case of rugged objects, since the appearance of them is sensitive to light changes and 3D rotation. Transparent objects change their appearance as pose and background changes, so a recognition rate of them decreases. Also, an appearance of objects with shiny parts usually is sensitive to a direction of light, light sources and even to simple affine transformations of the object. As a result of the conditions, it appears some light peaks and shadows on the surface of such objects and outside changes rapidly. As a consequence, recognition decreases rapidly, unless the same light peaks and shadows were present in the model template of the object.
The model should contain various poses of the object taken in diverse light conditions. Various poses stands for front side and backside of the object and all possible 3D rotations (off plane rotations) of it. On the other hand, translation, planar rotation (in plane, orthogonal to a camera) and small scaling (till the object does not change its appearance due to resolution change) are fully reconstructed by the library. Various light directions and sources change an appearance of the object, so the model should include images taken under a range of light conditions. Noisy and blurred images could decrease quality of the model, and should be removed from it.
The algorithm can recognize only rigid objects. At least significant part of the object should be rigid.