Blind Assistant

Blind Assistant is an attempt to realize a software that can help men and children that are blind or visual impaired. This software doesn't require a complex and dedicated hardware: in fact, it is designed to run on simple and cheap devices as Sony Playstation Portable (PSP).

Blind Assistant uses Nanodesktop tecnology for windows environment, it is able to speak with the user using the speech synthesis engine ndFLite, and it is able to recognizes the commands spoken by the blind through the speech recognition engine ndPocketSphinx.  In this way, the blind can control the program using the integrated voice interface.

At the moment, Blind Assistant provides 7 functionalities:

- Face recognizer

- Room recognizer

- Optical char recognizer

- Color scanner

- Mail reader

- DMTX scanner

- Clock/Calendar

 

The face recognition tecnology is provided using ndOpenCV, a porting of the original Intel OpenCV libraries that has been developed by the same author, and that is part of the Nanodesktop SDK.

The system is able to recognize  the name and the position of the people that are present in that moment in your room, using complex algorithms (as Intel EigenFaces/PentLand PCA).

Filippo Battaglia's version of ndOpenCV improves the Intel code because the algorithms have been modified in way to use EMI (Enhanced Mathematical Interface), an API integrated in Nanodesktop and written totally in assembly, that is able to use Sony VFPU to speed up the necessary computations. Using EMI, we can obtain the necessary CPU power.

Furthermore, Blind Assistant uses a client/server architecture with a second component called Blind Server, designed to run on a X86 PC, and that uses wi-fi connection to communicate with the client.

Through Blind Server, the mobile unit can execute calculus more complex than the  CPU power of the PSP could allow otherwise. Using BlindServer, the PSP can execute an image analisys: it can determinate the room where the blind is. During this elaboration, BlindServer uses the SIFT algorithm (Scalar Image Features Transform), a complex tecnology developed by Robert Hess and that is able to recognize the more significative points in an image.

Blind Assistant for PSP is able to read a simple text, too. The software for PSP integrates ndTesseract, a dedicated version of the OCR developed by HP and Google Foundation that is able to run on the MIPS processor of the Sony console.  ndTesseract can process an image of 1024x768 pixel within 2 minutes, obtaining a recognition rate of 80% in ideal condition.

As alternative to a local elaboration, Blind Assistant can use a particular communication protocol that allows to the client the interrogation of the Blind Server, via wireless. In the server, it is integrated a version for x86 processor of Tesseract: this method requires a separated laptop running the server and a wireless connection, but it is very faster (a response is received by the console within few seconds).

Blind Assistant integrates also a color scanner, that allows to the blind the recognition of a color in an object and a mail reader, a software that allows to receive mails and short messages. The software can interrogate periodically the POP3 server and, if new messages are received, a vocal message for advice is emitted. In a second time, the user can switch to mail reader mode, and it can download and listen the content of the new mails. Thank to support of the VMIME library, the software can manage both plain/text or HTML mails without problems. 

Starting from version CFW0006, BlindAssistant integrates also a clock/calendar and a scanner for DMTX matrices (http://www.libdmtx.org/) that allows to recognize labels placed on objects or obstacles.

The software supports the Sony GoCam webcam. It is required the PSP-2000 model, because of its 64 Mb of ram; however, a reduced version of the program for PSP-1000 (PSP-FAT) is provided too.

The author

 

The author of Blind Assistant is Filippo Battaglia, an electronic engineer graduated at University of Messina. I want to thank my teacher, the coordinator of Visilab, prof. Giancarlo Iannizzotto for the support and the guidance. I want also to thank Fabio Sfuncia for the support in the hosting of the HTML site. 

 

If you have troubles in Blind Assistant installation, you can write me to filbattaglia@libero.it or you can send a message to Nanodesktop official forum at www.psp-ita.com.

 

Download

 

This program works only on PSP custom firmwares. It has been tested (under PSP FAT and SLIM) with the custom firmwares version 3.71 M33-4, 3.80, 3.90 M33-3, and 4.01. The program is able to auto-update itself using the wireless connection.

Download Blind Assistant CFW006 (+ BlindServer 1.2)

License and terms of use

The source code is released under GPL license. You can use this program only for not commercial uses. See the file Blind Assistant License.Pdf in the package for further details.