Projects‎ > ‎Pen Projects‎ > ‎

STEM Binder

STEM - Science, Technology, Engineering and Math

Project Description

The Livescribe Pulse Pen is a commercial smart pen device that includes a video camera in its tip, a microprocessor that can run Java programs, memory, an audio speaker, a microphone, and a rechargeable battery. As the pen is touched to specially-printed paper, the camera “sees” a pattern of tiny dots (commercially known as the Anoto pattern) that encodes page and pixel coordinate data, and can deduce its position on a printed surface.  In the audio-haptic applications proposed here, the pen will speak the identities of any feature of a raised-line graphic that has been touched by the pen’s tip, and will permit complex interactivity with materials such as maps, curricula, games and reference tools.  Users can detect by touch not only raised surfaces but indentations such as channels, dimples and punched holes, subtle features not discerned by the fingers alone.  Users can first explore these images with their hands; then they can tap the pen’s tip to any feature to cause descriptive text to be played through the pen’s speaker or through connected ear buds. By this means, vision impaired readers can access complex pictorial material with good comprehension, and the raised-line graphic can also become the basis of a graphical user interface, allowing authors and artists to make highly interactive static images that communicate through multiple sense channels. 

Four applications:

  • Periodic Table. A complete reference tool on a single 11” x 17” page showing the periodic table of the elements.  Tapping places on the surface with the pulse pen allows the user to elicit several levels of audio information.
  • Scientific Calculator. A single-page application adapted from a calculator program developed for Livescribe. This application gives us chance to explore ways for embedding context-sensitive help features to simplify and speed up the process of performing potentially complex operations that require the user to press multiple keys in sequence.
  • Sudoku Game.  This popular pastime requires that users fill in grids of digits through trial and error, drawing on logical reasoning and spatial memory skills.  This application will provide insight into the development of multi-sensory presentation.
  • Biology diagram of a motor neuron.  This will allow the user to develop a concept of a three-dimensional physical object and to elicit information about it at several levels.

Image of Audio-tactile scientific calculator
Audio-tactile Scientific Calculator

Image of Audio-tactile motor neuron
Audio-tactile Motor Neuron