WearaBraille is a research project into wearable computing, and the use of MEMS sensors as novel input technology to mobile devices such as laptops, PDAs and smartphones. Our prototype device functions much like a Braille keyboard, but instead of buttons for each finger to press, there are accelerometers on the back of each finger, near the knuckles (keeping the palms and finger tips free). When a finger taps on a table top or other firm surface, the WearaBraille knows that the Braille dot associated with that finger is part of the current character. By tapping multiple fingers simultaneously to indicate multi-dot characters, the WearaBraille user can type text, as well as moving the cursor and performing other system control functions. The device connects to a computer or smartphone via a wireless connection, such as Bluetooth; it has been demonstrated with an iPhone, an Android smartphone, and a Windows laptop.
A story on CNET
mentions that Apple is looking at providing a general purpose protocol for accessibility accessories (joysticks, buttons and switches, and Braille devices) to connect to VoiceOver
on mobile devices running iOS (iPhone, iPad and iPod Touch). Specifically, the story links to a patent application
filed by Apple which describes how external peripherals could connect to VoiceOver on iOS (either wirelessly or through a cable) to provide access for "Users who are not physically present at a touch screen ... Users that are unable to provide touch input and/or are unable to view the screen of a touch screen".
These are precisely the use cases that WearaBraille
seeks to address. So far, we have interfaced with VoiceOver by pretending to be an existing commercial Braille Display device (whose interface protocol was publicly available), but if Apple publishes a general purpose protocol (API) for accessibility accessories connecting to VoiceOver, we could potentially support a much broader set of controls for the iPhone and iPad. We will be excited to see what controls this protocol provides.
When we started the WearaBraille project, we imagined that it would connect to smartphones as though it was a normal QWERTY Bluetooth keyboard. We wanted to minimize the need for software drivers or special apps, and we believed we could handle the conversion from Grade 1 Braille (actually, Computer Braille
) to the equivalent QWERTY characters. A lot of people asked about support for Grade 2 Braille
, but we didn't think it made sense to put that level of complexity into the device. We also weren't sure what operations beyond text entry and simple cursor movement we could support.
Last year, having been impressed by VoiceOver
on the iPhone, we saw that VoiceOver supports Bluetooth Braille displays (most of which include a Braille keyboard), and realized that we could talk to an iPhone in a different manner. After adding emulation of a Bluetooth Braille device to the WearaBraille prototype, we can now enter Grade 1 and
Grade 2 Braille; in addition, we can navigate between applications and fields on the iPhone, turn the volume up or down, and even answer a phone call! Check out the YouTube video on the main WearaBraille
We're looking at whether we can provide this level of access to an Android phone. We have written a simple app which supports Grade 1 entry to demonstrate the concept, but until Android has a comprehensive screen-reader, it doesn't make sense to build out complete support for Braille input. We are keen to collaborate with Android developers who are working on screen-reader applications.