Projects

doppellab frame
2010 – present
DoppelLab
Using a game engine to browse and interact with data from densely-deployed sensor/actuator networks

Sensor Networks
Dynamic Visualization
Cross-Reality

Screen Shot 2014-06-19 at 4.43.00 PM
2013 – present
Tidmarsh Living Observatory
A cross-reality browser to experience real-time sensor data through visuals and music

Sensor Networks
Dynamic Visualization
Cross-Reality

ChainAPI Architecture
2013 – present
ChainAPI
A hypermedia-focused API supporting request/response via HTTP and realtime via WebSockets.

Sensor Networks, IoT, Web Services

assembled
2013 – present
Tidmarsh Sensor Node
An extensible hardware platform for low-power wireless outdoor sensing applications.

Sensor networks

Circuit Stickers
2013 – present
Circuit Stickers
Circuit stickers are peel-and-stick electronics for crafting circuits.

Craft
technology for artistic expression

BoardLab
2013 – present
BoardLab
A handheld probe that allows one to query and annotate the design data for a circuit board by direct interaction with the circuit board.

Human Computer Interaction, Fabrication

GrassRootsMobile
2012 – present
GrassRootsMobile
A GSM payments enabled powerstrip to enable sharing of electricity via informal connections.

Energy Management


Header32012-2014
PrintSense
A multimodal on-surface and near-surface sensing technique for planar, curved and flexible surfaces. This technique leverages temporal multiplexing of signals coming from a universal interdigitated electrode design, which is printed as a single conductive layer on a flexible substrate.

Sensor Networks
Flexible, printed electronics


coverHP2012-2013
Cuttable Sensors
We propose cutting as a novel paradigm for ad-hoc customization of printed electronic components.  A technical evaluation compares different topologies and shows that the sensor remains functional when cut to a different shape.

Sensor Networks
Flexible, printed electronics

GESlideLocation
2012 – present
Gestures Everywhere
A multimodal framework for supporting ubiquitous computing, aggregating the real-time data from a wide range of heterogeneous sensors.

Ubiquitous Computing, Gesture Recognition, Interactive Displays

FreeD frame
2011 – 2014
The FreeD
A handheld digital milling device for craft and fabrication

Ubiquitous Computing
Human-Computer Interaction
Digital Fabrication
Craft

Device on hand
2014 – present
Wrist gestures
Always-available wrist-worn gesture recognition

Ubiquitous Computing, Gesture Recognition

wristque_1
2011-2013
WristQue
A personal wearable sensor wristband that enables the wearer to take control of nearby devices through automatic sensing and a pointing-based user interface.

Wearable sensing, building control, human interfaces

althumb
2010 – present
Adaptive Lighting
We have constructed a variety of lighting networks to optimize the electrical and photometric properties of solid-state lighting while meeting the illumination levels set by the user.

Sensor Networks
Energy Management
Control & Optimization

RingTag
2010
RingTags
Passive Ring-like gestural finger controller for a wearable computer system..

Passive Sensing
Wearable Computing
HCI

Screen-Shot-2013-10-21-at-7.42.23-PM
2010-2011
Inkjet printed floor sensor
A floor sensing system that is based on a new conductive inkjet technology, which allows capacitive sensor electrodes and different types of RF antennas to be cheaply printed onto a roll of flexible substrate that may be many meters long.

Passive Sensing
Flexible, printed electronics, Floor sensing

boxie
2009 – 2010
Interactive Story Capture Camera
A robotic camera programmed with an interactive scenario designed to pull stories from people, captured as structured media streams.

Human-Robot Interaction
Social Computing
Interactive Media

portable thumb
2008 – 2009

Personalized Building Comfort Control

A novel air-conditioning control system, focused around the individual, with body worn sensors determining comfort in real-time. Multi-modal actuation and sensing allows the building to respond to the user wherever he or she is located. Issues of arbitration between users and comfort/cost tradeoffs are addressed.

Sensor Networks
Building Energy Systems
Low-Power Circuit Design
Reinforcement Learning

Portals Thumb
2008 – 2010

Ubiquitous Sensor Portals

A multimodal sensor/actuator network installed throughout the Media Lab to support a variety of projects.

Sensor Networks
Ubiquitous Computing
Pervasive Dynamic Media
X Reality


2008 – 2010
NONO Badge
We are constructing a configurable infrastructure to protect users’ dynamic levels of privacy in a pervasive sensor network. The work is based around a configurable privacy badge that can alert the user to the presence of participating sensor networks, plus emit an RF beacon with which the network can gauge the level of privacy desired.

Sensor Networks
Ubiquitous Computing
Privacy


2007 – 2008
Ubicorder

The Ubicorder is a mobile, location and orientation aware device that enables users to browse and interact with real-time data from sensor networks.

Sensor Networks
Ubiquitous Computing
Mobile Devices

ChainMail Thumb
2008

ChainMail

An ultradense multimodal sensor network as an electronic sensate “skin”.

Sensor Networks
Electronic Skin
Tactile sing

SportSemble Thumb
2006 – 2013

sportSemble

A high speed wearable wireless sensor network that enables direct measurement and analysis of the extreme forces that an athlete’s body experiences during activity. These measurements allow sports medicine doctors to understand, treat and prevent sports injuries.

Sensor Networks
Biomechanics
Sports Medicine

Spinner Photo
2006 – 2010

Spinner

Application of a lab-wide sensor network platform designed to detect and capture fragmented events of human behavior that can be collected and sequenced into a cohesive narrative that conveys a larger overall meaning. This project also looks at the development of parametric models of narrative that can be mapped on to sensor-detectable elements of human activity.

Human-Centric Sensing
Video Networks
Phenomenology
Experiential Design
Narratology

PuDAC-Demo Thumb
2007 – 2008

PuDAC

Interactive music running on a Nokia 800 (phone-like PDA) exploiting a Pure Data Compiler (PuDAC) and wireless wearable sensors for mobile interactive music and exercise-generated media – a glimpse at the future of the portable music player.

Sensor Networks
Interactive Music
Mobile Devices

CargoNet Tag
2006 – 2008

CargoNet

Micropower sensate tags for supply-chain management and security applications. These wireless sensor nodes are low-cost devices that monitor many modalities relevant to shipping and supply-chain needs. They keep extremely low standby current, asychronously waking up to exceptional phenomena with dynamically adjustable thresholds.

Sensor Networks
Power Management

Tricorder
2006 – 2010

Tricorder

Star Trek style Tricorder that lets you ‘See through the walls’ by using multi hop sensor networks. An Augmented Reality platform for sensor networks.

Sensor Networks
Augmented Reality

Dual             Reality Lab
2006 – 2010

Dual Reality Lab

Virtual and real worlds that reflect, influence, and merge into each other by means of deeply embedded sensor/actuator networks.

Sensor Networks
X Reality

Plug
2005 – 2008

Plug

A power strip imbued with sensing, computational, and communication capabilities in order to form the backbone of a sensor network in domestic and occupational environments.

Sensor Networks
Ubiquitous Computing

SAW Device Diagram
2005 – 2007

uTags

Development of a low-cost system to track the precise (cm-scale) 3-D position of large numbers of objects tagged with passive SAW microwave RF transponders at short-range (3-100m) and in real time.

Ubiquitous Computing
Mobile Sensor Networks
Localization

Decision Tree
2005 – 2008

Groggy Wakeup

An automated framework for generating a population of classifiers for power-efficient detection in wearable sensors.

Power Management
Adaptive Systems
Wearable Computing

Parasitic Hopping Robot
2003 – 2004

Parasitic Mobility

Research into devices that can use the actuation and navigational intelligence of hosts present in the environment.

Power Harvesting
Mobile Sensor Networks
Robotics

Adaptive Music Controller : Wailing!
2003-2004

FlexiGesture

A sensor-rich adaptive gesture and affordance learning platform for electronic music control.

Adaptive interfaces
Multi-DOF sensing
Electronic music control

Badge
2003-2007

The UberBadge

A versatile wearable computer platform in the form of a badge, featuring three processors, RF and IR communication, built in microphone, audio output, large LED display and LCD capability, pager vibrator, large flash memory capacity, and extensive expansion possibilities. Designed to enable a wide variety of applications where mobile and social computing intersect.

Wireless Sensor Clusters
Wearable Systems

A compact RF transceiver card, originally developed for the UberBadge project, but useful for many different applications.

Wireless Sensor Clusters
Wearable Systems

Cooktops
2002-2003

Sensate Cooktops

In association with TTT sponsor Schott Glass, this project explores various techniques for transduction in glass ceramic cooktops. We have developed a capacitive slider for sensing continuous fingertip position across the glass, several techniques of sensing the presence and size of pots above the glass, haptic response to capacitive button activation, micropower wakeup of cooktop electronics with quasipassive sensing, and remote measurement of glass temperature above a burner via IR and acoustic monitoring.

Sensor Architectures
Interactive Surfaces

Trible
2003-2004

Trible
The “Trible” (Tactile Reactive Interface Based on Linked Elements) is an excursive in dense distributed sensing, computation, and actuation. It has the form factor of a ball, tiled with a multimodal sensate “skin” consisting of 32 networked elements. Each tile measures pressure at 3 locations, temperature, sound via a microphone, illumination, and dynamic tactile stimulation with up to 12 channels of protruding, touch-sensitive “whiskers”. Each tile can also respond with a small audio speaker, a pager vibrator, and a bright RGB LED. There is no central processor – each tile talks to its neighbors through conductors in the frame. The Trible is a research platform for the application of decentralized control and distributed estimation to human-computer interaction.

Sensate Media
Perceptive Objects

Pushpin Computing
2002-2004

Pushpin Computing

Small bottlecap-sized computers, each with two “thumbtack-like” rear-protruding pins, draw their power from a layered composite into which they are pushed and communicate with neighbors via infrared or capacitive coupling. Provisions are made for simple sensors to be easily added to each pushpin, enabling the system to serve as a testbed for developing algorithms for extremely high-density, smart sensor networks.
Sensate Media
A compact, configurable wireless sensing system, for which several sensing boards (e.g., tactile, inertial, sonar, etc.) have been designed. These boards can be stacked in any order and configuration atop a main processor/RF board, allowing the sensor suite to be easily customized. A TDMA polling scheme enables multiple stacks to be used simultaneously. Although it has been primarily designed for wearable applications, this device serves as a general platform for compact multimodal sensing.
Sensor Architectures
Wireless Sensor Clusters
Wearable Systems
Gait shoe
2002-2005

The Gait Shoe

By measuring many (e.g., 20) different parameters at each foot using our Sensor Stack, these shoes become a wearable gait laboratory. Data is analyzed for features indicative of particular gait problems. This system, in collaboration with the MGH Biomotion Laboratory, is intended to diagnose gait conditions and provide real-time corrective feedback to the patient.
Embedded Healthcare
Wireless Sensor Cluster
An sensorcentric object model for distributed systems.
Wireless Sensor Clusters
Sensate Media
This project explores microwave sensing for detecting noncontact gesture. Several systems have been developed and deployed, including very-low powered Doppler motion-sensing radars with onboard digital feature extraction and modified swept-Doppler ranging radars.
Sensor Architectures
Perceptive Spaces
ZTiles
2000-2004

ZTiles

In collaboration with the University of Limerick a sensate floor is made from networked sensor tiles, each of which has over a dozen small pressure sensors connected to an embedded computer. All tiles are networked when they are snapped together. Local communication between tiles establishes an ad-hoc network. Sensor signals are compared across tiles, enable stimuli from dynamic sources, such as footsteps, to be clustered and abstracted into a basic set of parameters, which are routed peer-peer across connected tiles to an external computer.
Sensate Media
Expressive Interfaces
Perceptive Spaces
Precision capacitive sensing is used to determine the dimension of an actively-controlled nanometer-scale gap. This project, a collaboration with Alex Slocum’s group in MIT’s Mechanical Engineering Department, has many applications in biochemistry and nanoscale chemistry.
Sensor Architectures
A flashlight with an optically coded beam quasi-passively wakes up active tags sleeping at extremely low power drain. If the tag’s ID matches the interrogating beam’s request, a tag-mounted LED flashes. The compact tags can be mounted on the edge of removable media, for example, allowing a particular item on a crowded shelf to be found by merely scanning the flashlight across. The metaphor of seeking real-world objects with a flashlight is thus extended into the realm of data.
Micro/Self Power
Perceptive Spaces
Wireless Sensor Clusters
Knocks and taps across a large sheet of glass are detected by a set of pickup transducers, then characterized by their frequency content and located via differential timing. Essentially any single-paned window can be converted into the equivalent of a touch screen by this inexpensive technique, with no hardware mounted outside the glass.
Interactive Surfaces
Perceptive Spaces
Expressive Interfaces
An extremely low cost and compact wireless motion sensor sends out a brief RF pulse when jerked. As this device is very inexpensive, it can be given away as a “ticket” to an event, allowing large audiences to kinetically interact with responsive media. Feature extraction algorithms and content mapping strategies to exploit the human “schooling” and self-organizing reflexes with this system. Although our research looks at interactive dance as an application, other uses (e.g., interactive gesture and games at sports events, conventions, and large concerts, very low-cost, wireless shock monitoring of packages, etc.) abound.
Wireless Sensor Clusters
Expressive Interfaces
Micro/Self Power
A Swept-RF magnetically coupled resonant tag reader has been developed that exhibits enhanced stability, high resolution, and the ability to address up to three reader coils. This enables an ensemble of tagged objects to be wirelessly tracked in 3 dimensions. This device has been used in a musical environment that explores high-level gestural control of musical structure (e.g., the definition, manipulation, and overdubbing of musical sequences and arpeggiation) in live performance.
Expressive Interfaces
Magnetic Tag Tracking
Magic Carpet
1997-2002

The Magic Carpet

An interactive space has been designed that consists of a carpet atop a mesh of piezoelectric wire (tracking foot position and dynamic pressure) and a pair of Doppler microwave motion sensors (to respond to movement of the arms and upper body). This system has been used for several immersive musical installations, and is now a permanent installation at the MIT Museum.
Perceptive Spaces
Interactive Surfaces
Expressive Interfaces
This project has resulted in a batteryless pushbutton that wirelessly transmits a 12-bit digital code to the vicinity (e.g., 25-100 feet, depending on the RF environment) when pressed. This enables digital controls, from light switches to garage door openers, to be embedded essentially anywhere, without worrying about battery life or wiring.
Micro/Self Power
Wireless Sensor Clusters
A very low cost active magnetic tag tracker has been developed to track the position and absolute orientation of a PDA around a set of reader coils. Three such tracking stations have been built and installed for the “Atmospheres” installation at the MOMA “Workspheres” design exhibition.
Magnetic Tag Tracking
Expressive Interfaces
Nova
2000-2001

The Terminova

Free-gesture musical controllers suffer from the drawback of having no tactile (and often no visual) reference for their player, hence they are difficult to competently master. This project has developed a hybrid musical controller, consisting of a Theremin-like, free-gesture capacitive hand sensor in the horizontal axis, punctuated by an array of vertically-directed rangefinding lasers on mechanical carriages that can be moved to preassigned positions. The lasers, in light fog, are seen as references (or frets) – the position of the hand is also detected along the laser beams, affording another channel of expressive control.
Expressive Interfaces
Media Matrix
2000-2001

Media Matrix

This project draws on ideas from distributed, embedded, and parallel computing in order to address the creation and management of databases composed of large collections of physical objects (e.g. mini DVs in a rack or books in a library). An IR interrogation beamed at one object is transferred between neighboring objects peer-peer, diffusing to all objects in the environment (e.g., in the bookshelf). The sought-after item then illuminates an indicator.
Sensate Media
Perceptive Spaces
This project is a design study for a TDMA basestation to be used with very low power CMOS digital transmitters developed at MIT’s MTL for wireless sensor systems.
Wireless Sensor Clusters
Swept Tags
1998-2001

Swept RF Tagging

Several reader systems have been developed to wirelessly track the position of magnetically-coupled resonant tags. By making the tag’s resonance frequency a function of a local parameter (e.g., pressure or controller displacement), the tag can also become a sensor. These systems have been used to develop several demonstrations of “tangible bits” (where simple objects become controllers), the most engaging of which is a multimodal music environment called “Musical Trinkets.”
Magnetic Tag Tracking
Expressive Interfaces
Laser Wall
1997-2001

The LaserWall

An inexpensive scanning laser rangefinder has been developed to track bare hands against a large interactive display. This system provides a simple retrofit to any large surface – by putting the rangefinder in one corner, a large virtual “touchscreen” is created, with hand tracking insensitive to background light. This system has been used in public with several interactive applications ranging from information browsers to musical interfaces.
Interactive Surfaces
Expressive Interfaces
An inexpensive, compact package has been developed to monitor traffic (magnetically sensing passing cars) and detect local road conditions (e.g., temperature and moisture). A summary is wirelessly broadcast to a receiver mounted above the road within circa 300 meters of the sensor package. As the specified battery is projected to last for over a decade (depending on conditions), a network of these low-cost packages can be embedded into the roadbed, allowing the traffic report to be generated right in the street.
Wireless Sensor Clusters
Perceptive Spaces
A simple scripting framework has been created to recognize combinations of simple “atomic” gesture components in data from body-mounted sensors. Much as the way in which phonemes combine to form words, the user is able to specify a script of sequential microgestures that form a desired macroscopic gesture. When sensor data makes a good match with a specified script, the gesture is detected. This algorithm is extremely efficient and is able to run in real time on a low-end PDA, for example. A pair of small, wireless, handheld, 6-axis inertial measurement units (as shown here) were constructed to capture gesture and test this framework.
Wireless Sensor Clusters
Perceptive Objects
A compact suite of medical instruments has been integrated into a kit with a connected palmtop computer. Software has been written to guide a medical assistant through the process of running the needed tests on a patient, collecting the required data, and transmitting it to a remote medical facility for diagnosis. This system has been used with the LINCOS installations to provide medical care in rural Costa Rica.
Embedded Healthcare
Wireless Sensor Clusters
Electricity is generated while walking by flexing and pressing compact piezoelectric elements unobtrusively embedded into a shoe sole. As a demonstration, a self-powered wireless digital ID code is transmit directly from the shoe after every few steps.
Micro/Self Power
Wearable Systems
A suite of 16 diverse sensor channels is embedded into each of a pair of shoes. All parameters are wirelessly broadcast directly from the shoes to a nearby basestation with 50 Hz full-state updates. A very early example of compact multimodal wireless sensing, this system has been designed and used for interactive dance. With so many degrees of freedom measured, even a simple musical rulebase enables a dancer to produce an engaging sonic accompaniment.
Wireless Sensor Clusters
Expressive Interfaces
Wearable Systems
A wideband, dispersionless monopulse acoustic receiver can theoretically be constructed with continuously-tapered, co-located subapertures. Such a device would have the capability of locating the direction of arrival of sonic transients by calculating a simple sum and ratio. This project has built and characterized such a device.
Sensor Architectures
A large mylar balloon has been bonded to a piezoelectric foil sheet so it can act as both a microphone and a loudspeaker. A set of electronics has been designed for the balloon to detect significant audio activity, and play a selection from a set of prerecorded audio samples when the activity ceases,essentially enabling one to have a “dialog” with the balloon. Over 50 such balloon systems were produced and installed all over the Weisner Building for the Media Lab’s 10’th anniversary celebration.
Perceptive Objects
Sensor Architectures
An 16-element array of 4 x 4 programmable electrodes was built in order to study electric field imaging in air. Each electrode could dynamically be assigned to either transmit or receive.
Sensor Architectures
Many different musical controllers (ranging from digital batons to sensor chairs) were designed and constructed for the “Brain Opera”, a very large interactive musical installation that was produced at the MIT Media Lab and toured throughout the world.
Expressive Interfaces