A new prototype can transform a notebook into a notebook computer, a wall into an interactive display, and the palm of your hand into a smart phone display. Researchers at Microsoft and Carnegie Mellon University say their new shoulder-mounted device, called OmniTouch, can turn any nearby surface into an ad hoc interactive touch screen.
OmniTouch works by bringing together a miniature projector and an infrared depth camera, similar to the kind used in Microsoft’s Kinect game console, to create a shoulder-worn system designed to interface with mobile devices such as smart phones. Instead of relying on screens, buttons, or keys, the system monitors the user’s environment for any available surfaces and projects an interactive display onto one or more of them.
OmniTouch does this automatically, using the depth information provided by the camera to build a 3-D model of the environment. The camera acquires depth information about the scene by emitting a patterned beam of infrared light and using the reflections to calculate where surfaces are in the room. This eliminates the need for external calibration markers. The system rebuilds the model dynamically as the user or the surface moves—for example, the position of a hand or the angle or orientation of a book—so the size, shape, and position of these projections match those of the improvised display surfaces. OmniTouch figures out what’s in front you and fits everything on to it.
The system also monitors the environment for anything cylindrical and roughly finger-sized to work out when the user is interacting with it, again using depth information to determine when a finger or fingers make contact with a surface. This lets users interact with arbitrary surfaces just as they would a touch screen,. Similarly, objects and icons on the ad hoc “screens” can be swiped and pinched to scroll and zoom, much like on a traditional touch screen. In one demonstration art application, for example, OmniTouch used a nearby wall or table as a canvas and the palm of the user’s hand as the color palette.
There’s a growing interest in the area of having ubiquitous, intangible displays embedded in the environment. And although new generations of smart phones tend to have increasingly higher-quality displays ,users would be willing to put up with lower-quality projected images, given the right applications.