A Multimodal Tablet–based Application for the Visually Impaired for Detecting and Recognizing Objects in a Home Environment

Rabia Jafri and Syed Abid Ali. Proceedings of the 14th International Conference on Computers Helping People with Special Needs (ICCHP 2014), July 9-11, Paris, France, Springer-Verlag Berlin Heidelberg, Lecture Notes in Computer Science (LNCS), Volume 8547, pp. 356-359, 2014.

Abstract: Object recognition solutions for the visually impaired based on a single modality cannot provide optimal performance under all circumstances, since each modality is best suited for particular usage scenarios. An object recognition application for the visually impaired, meant for a RFID-enabled tablet, which combines three approaches – RFID-based, visual-tag computer vision based and non-visual tag computer vision based – into a single piece of software is, therefore, presented in this paper. This solution has the benefits of being portable, accessible, low cost (the user needs only an RFID-enabled tablet, some inexpensive passive RFID tags and some visual tags (which can be printed out for free)) and more robust in a wider range of conditions than the approaches it is comprised of. The application will be adapted to other mobile platforms and devices (e.g., RFID-enabled smartphones) in the future.