September 2003–October 2006
Natural Interaction enables a novel form of interaction with software, making it possible to describe things by sketching, gesturing, and talking about them in a way that feels completely natural, yet have a computer understand the messy freehand sketches, casual gestures, and fragmentary utterances that are part and parcel of such interaction. Once the sketch is understood, the information it contains can be handed off to other applications for simulation, design checking, design completion, or refinement.
Natural Interaction:
- has been implemented in a variety of domains, including simple 2-D mechanical devices and UML software diagrams
- deals with the noise inherent in hand-drawn sketches
- uses a variety of knowledge sources to resolve ambiguity
- provides modeless interaction for both sketching and editing
- allows ideas to be evaluated early in the design cycle
Natural Interaction
Natural Interaction will let computer tools capture and understand ideas that are today captured in pencil on scraps of paper. This capability can introduce design capture into the earliest stages of the design process and radically shorten the design cycle. MIT’s Design Rationale Group is working to imbue a wide variety of surfaces with natural interaction capabilities, ranging from active whiteboards, to tablet computers, to desktops in classrooms and benches in laboratories. This vision of computing spans the range from tablet computer notebooks full of Natural Interaction to a new concept of desktop computing: your (physical) desktop should compute.
Exploring and Using Natural Interaction (Magic Paper)
- View video of Natural Interaction in use
- Download prototype software for sketching 2-D mechanical devices
- Read or download papers describing the multiple facets of ongoing work on the project, including speech and gesture understanding, and multi-modal integration.
- Learn about the goal of natural interaction
Investigator: Prof. Randall Davis, Dept. of Electrical Engineering and Computer Science