Multispectral Imaging Leads to Advances in Eye Tracking

Multispectral imaging allows for the identification, classification, and segregation of objects and areas in scenes based on their spectral signatures. In her doctoral dissertation, at the University of Eastern Finland, Ana Gebejes, used the FluxData 1665-MS7, in a novel application. She created spectral videos for use in eye analysis and tracking. In addition to the proof of concept, she created a publicly available database of spectral images and videos, the Spectral Eye vidEo Database (SPEED). This database can be used by researchers for creating and testing new methods of dynamic eye analysis. The database contains both still images and videos of eyes performing different eye-tracking tasks under different conditions, such as wearing corrective lenses or sunglasses and different lighting conditions.

Due to recent technological advances in optics and electronics, spectral-video sensors are now able to capture spectral information at video rates. Dr. Gebejes carefully analyzed the spectral, temporal and spatial characteristics of the FD 1665-MS7, a seven-channel spectral video system, and found it was uniquely suited for eye tracking. Not only were the spatial and temporal resolution high enough capture high resolution images of the fast-moving eye, but the spectral characteristics of the camera allowed excellent spectral classification of the different parts of the scene. Using spectral segmentation algorithms, she could identify features in the image such as the pupil, iris, blood vessels, sclera, skin and eyebrows.  

In spectral videos, each frame of the video contains a spectral image cube, allowing spectral video systems to capture spatial, spectral, and temporal information simultaneously.  In the figure, below, each column represents an image cube. Each row shows the response for one of the seven channels for five frames of a video arranged in columns. For each pixel in the image, a spectral signature can be calculated. With knowledge of the scene illumination, identification and segmentation of the scene can be achieved.

Pictured above is five selected frames from one standard video taken from FluxData's 1665-MS7

Compared to traditional eye-tracking methods that use near-IR imaging mounted on a wearable camera system, spectral imaging using the FD 1665-MS7 makes it possible to use a remote camera system. This can allow for use in many more applications from the operating room to security and surveillance.  Traditional eye-tracking relies on spatial scene analysis to locate the pupil. However, spectral analysis of the scene allows for better segmentation of the pupil under harsh conditions, for example, when the subject is wearing glasses, where traditional pupil-detection algorithms fail.

Dr. Gebejes describes her research as a first step in creating the new eye-tracking technologies of the future. She believes that her results provide evidence for the utility of using spectral data for eye-tracking. The FD 1665-MS7 can provide for real-time scene analysis and future work may focus on the most efficient methods to accomplish this for eye-tracking. The data in the SPEED database has the potential to be used in other areas of eye-related research including medicine, biometric, and vision research.

To read the full dissertation by Ana Gebejes, click here.