Tangible Interfaces and Graspable Interfaces

This page gives a access to web ressources on Tangible User Interfaces and related approaches (Graspable Interfaces, Tangible Interaction, Mixed Reality). It is limited mostly to web pages and project or persons home pages. If you know something which is missing here, please tell me.
Outdated links which I'd like to keep for historical reasons or links which might show up again (or be refound)
                are shown in dark red ....

What are Tangible Interfaces - Definitions
Projects, People and Research Groups - start
        Paper-based Tangibles or Interactive Paper
        Self-Describing Building Blocks - 3D Modelling
        Tangible, interactive Toys - Games & Toys
        interactive music or composing instruments - Music Applications
        Design & Interaction Design Approaches to TUI-Design - Design
        Mixed & Augmented Reality and Tangible Interaction - MR, AR and more
        Technology Stuff: Technology
Bibliographies and other ressources: Ressources
Author contact

NEW: Special issue of Personal and Ubiquitous Computing journal: Tangible interfaces in perspective
          November 1, 2003: Paper submission deadline (closed)

Toolkit Support for Interaction in the Physical World Workshop at Pervasive 2004, 20. April in Linz/Vienna (workshops in Vienna). Deadline for submissions 26 January

What are Tangible Interfaces ?

Fitzmaurice on Graspable UIs in his thesis:
A Graspable UI design provides users concurrent access to multiple, specialized input devices which can serve as dedicated
physical interface widgets, affording physical manipulation and spatial arrangements. Hence input control can be "space-multiplexed."
That is, different devices can be attached to different functions, each independently (but possibly simultaneously) accessible. This, then
affords the capability to take advantage of the shape, size and position of the physical controller to increase functionality and decrease
complexity. It also means that the potential persistence of attachment of a device to a function can be increased. By using physical objects,
we not only allow users to employ a larger expressive range of gestures and grasping behaviors but also to leverage off of a user's innate
spatial reasoning skills and everyday knowledge of object manipulations. These physical artifacts are essentially "graspable functions" -- input devices which can be tightly coupled or "attached" to virtual objects for manipulation, or for expressing actions. These artifacts need to have spatially-aware computational devices.

Ullmer & Ishii on Tangible Interfaces (this definition is the most wide spread):
Generally graspable and tangible interfaces are systems relating to the use of physical artifacts as representations and controls for digital information. A central characteristic of tangible interfaces is the seamless integration of representation and control, with physical objects being both representation of information and as physical controls for directly manipulating their underlying associations. Input and Output devices fall together.
There are 4 characteristics concerning representation and control:
  1.Physical representations are computationally coupled to underlying digital  information.
  2.Physical representations embody mechanisms for interactive control.
  3.Physical representations are perceptually coupled to actively mediated digital representations.
        (visual augmentation via projection, sound...)
  4.Physical state of tangibles embodies key aspects of the digital state of a system.
    (TUIs are persistent: turn off the electrical power and there is still something meaningfull there what can be interpreted)
And: (this should be heaved up to be characteristics as well)
- Tangible interfaces rely on a balance between physical and digital representations.
   Digital representations are needed to mediate dynamic information.
- The elements of TUIs are spatially re-configurable (in contrast to tangible digital appliances) (pointed out in Brygg Ullmers PhD-thesis)

From the thesis of Volker Brauer (artec) about "Gegenständliche Schnittstellen"
(an area combining some Augmented Environments and Tangible Interfaces, thus the description also applies to some Augmented Environments which offer a shared environment and natural interaction)
Physical spatiality describes the co-presence of user, objects and other users in one interaction space. This space is a hybrid. Physical objects have a double affiliation to real/physical and virtual/digital space, but must still obey laws of the physical world. Because of co-presence of users and objects, interaction takes place IN the user interface. Therefore input and output space coincide. The user experiences a bodily shared space, his/her body is in the same space as the interaction objects.
Haptic directness denotes direct manipulation where the physical, graspable objects themselves are the interface. The user has direct contact with the interface elements Interaction is unmediated and intuitive, leading to ‘direct engagement’. Unmediated, direct manipulation results in isomorphic and structure-preserving operations.

What does it mean for an object to be ‘graspable’?   (from one of my own papers)
It means that it is of material nature, following physical laws, is situated in an environment and can be experienced by the living body. One interacts directly with graspable objects, touching and feeling them. The word tangible expresses the doubleness of touch -- being touched by the same thing that one touches, being active and passive at once.  Grasping refers to the act of enclosing something with ones digits. Our hands are the most sensorised part of our body and their representation occupies a rather large part of our brain. They have been our foremost and most versatile, creative instruments for ages until technology reduced their utilisation to clicking and pointing.

Projects, People and Research Groups:

Credits should be given to John Frazer and Robert Aish, who were the first to (in parallel) develop the idea and first implementations of a tangible construction kit for creating virtual models at the end of the 70s and beginning of the 80s. This work, coming from the areas of architecture and CAD, has only recently been rediscovered by people interested in tangible interfaces.

George W. Fitzmaurice  together with Hiroshii Ishii and William Buxton 1995 introduced Bricks, tangible new input devices that allow direct control of electronic or virtual objects through physical handles for control, and coined the term "Graspable User Interface".
 Fitzmaurice dissertation defines and explores Graspable User Interfaces,  putting emphasis on the fact that input control can then be "space-multiplexed", that is different devices can be attached to different functions, each independently (but possibly simultaneously) accessible. This allows a larger expressive range of gestures and grasping behaviors but also to leverage off of a user's innate
spatial reasoning skills and everyday knowledge of object manipulations. 2-handed interaction was another major point in this thesis.

Real Reality is a specific approach highly related with Tangible Interfaces and was developed at the research center artec in Bremen (now: artecLAB).
Projects around Real Reality have been: RUGAMS, Eugabe, BREVIE
Real Reality focuses on bridging between two different modelling worlds - real physical world and virtual worlds, and on the use of functional physical models (that is models having real functionality, instead of only using bricks or blocks without any behavior).
DERIVE  extended the concept and mixed it with ideas from mixed environments to enable distributed, mixed modelling of pneumatic circuits. HyperTubes connect real parts of a circuit with purely virtual, simulated circuits and with other real circuits which are in another part of the world.

My own research focuses on the use of tangible interfaces in cooperative & collaborative settings, esp. for designing and problem-solving: Look at my publications on analysis of group interaction with tangible media and design issues related to this issue.

MIT's Tangible Media Group  (led by Hiroshii Ishii) focuses on Tangible Interfaces and introduced the term "tangible user interfaces".  Most of their projects  and publications are relevant for Tangibles, starting with the MetaDesk, and eg. Triangles, URP, MediaBlocks, IlluminatingLight, IlluminatingClay.... A related approach pursued by this group is Ambient Media, which can be explained as using physcical objects and environments as display device, devoid of the input/manipulation facility of  Tangible Interfaces.....

There are several other MIT Groups with projects  related to Tangibles:
Hyperinstruments Group,
The Lifelong Kindergarden Group: Programmable Bricks, Beyond Black BoxesToys to think with (e.g programmable beads)
Gesture & Narrative Language Group: Creating stories with a blank-page book and wooden animals:  AnimalBlocks
        Embedding story telling functionality into toys:  Sage, Rosebud (link lost), SAM, Storymat
Synthetic Characters Group: the Swamped Project - Using Plush Toys to Direct Autonomous Animated Characters

Tangible Viewpoints  a system for interacting with a multiple point-of-view story, by Ali Mazelek The interface uses wirelessly sensed graspable pawns on a top-projected surface to navigate through the story. When a pawn is placed on the sensing surface, the story segments associated with its character's point-of-view are projected around it and can be interacted with.

The Origami Desk  is an interactive desk with sensing and projecting, where users learn to fold paper into Origami shapes. The desk shows videos that demonstrate what the hands should do, projects  lines onto the paper indicating folds, and  monitors the paper folding to give feedback. Origami Desk utilizes projection, electric field sensing, and  low-cost radio-frequency identification tags integrated into the paper

Datatiles:  Jun Rekimoto at Sony worked on one project with Brygg Ullmer of MIT's Tangible Media Group
 Brygg Ullmer now is at ZIB in Berlin and started experimenting with using tangible interfaces for scientific visualisation. 

Center for Lifelong Learning & Design (L3D) in Boulder, Colorado :
The EDC(Envisionment & Discovery Collaboratory) is a system to support community participation in public planning and design. There are now 2 versions, an (older) SmartBoard version, based on a touchscreen hardware, and (new) the PitA-BOARD which is based on a chessboard-like grid, which can detect tokens. The system is inspired by the game board of physical design games, but augments the tactile aspects of physical game pieces with dynamic capabilities of compuational simulation.
When I visited at the L3D lab, we did an exploratory assessment of both systems - resulting reporting publications can be found on my publications page  (DIS 2002, PDC 2002). Research on the EDC is pursued by Gerhard Fischer, Hal Eden, Eric Scharff, Ernie Arias and other members of L3D.
- new PitA-Board Website

Hypergami has been developed by Mike and Ann Eisenberg from L3D. These are software environments for  designing and building paper sculpture using polyhedra and custom variants of polyhedra. In some ways, this is the other way around - creating something physical by interacting with the virtual....

Masanori Sugimotos Interaction Technology Laboratory (unfortunately mostly in Japanese...) worked with L3D and uses similar technology to L3D's PitA-Board for an educational game in environmental issues for children

TICLE  (Tangible Interfaces for Collaborative Learning Environments): A tangible interface enable computer systems to
"watch" as children work together on puzzles and other educational tasks in a physical environment. The system may then act as a "guide on the side". System developed for the Goudreau Museum of Mathematics in Art and Science, where visitors can play with 22 mathematical puzzles.

Cubicle - Tangible cuboid interfaces: a multifaceted, multi-sensory wireless tangible input device

Interactive Institute, Sweden: started some Tangible Interfaces Projects, e.g Tags
The Narrative Toys Group at Interactive Institute with Psst! the Programmable SoundScape Toy, the Communicating Dolls, BeastyBox.

Candle Altar The Creation of an Artifact to Facilitate Spirituality 

DMG (Design Machine Group) researches how the digital change effects design.Subsection on "physical computing"
Navigational Blocks (4 wooden blocks to interact with a history information booth of Seattle's Pioneer Square) and
the Tangible MouseHaus
(physical front end to pedestrain behavior simulation)
Another site on MouseHaus Table for supporting discussion and participation in urban planning . Unlike other tangible interaction projects, MouseHaus enables users to employ ordinary materials in the interface. Users register objects to represent urban design elements by showing them under the camera, then they use the objects to construct a street layout for simulation.

Paper based Tangibles resp. Interactive Paper

Wendy Mackay: did several projects on Augmented resp. Interactive Paper, one could also term it "paper-based tangibles" (Video Mosaic - paper storyboards to control on-line video editing system, Ariel - paper engineering drawings as  interface, Caméléon - flight strips in air traffic control). Her work combines detailed ethnographic work place studies, researching existing work practices around artifacts, and developing technology for these settings.

The Designers Outpost A task-centered tangible interface for web site information design, that combines the affordances of paper and a large physical workspace with the advantages of electronic media to support the early phases of collaborative information design for the web.

RASA: a computer augmented environment for military command and control, augments a map and paper scraps and allows multimodal interaction (speech, handwriting, gesture)

Paper++ project aims to develop and assess innovative concepts and systems enriching the use of paper in everyday settings; the home; the classroom and other public spaces.

Pulp Computing project (Paper based interfaces to digital media) at HP Labs, Palo Alto, Nomadic Computing Department aiming to integrate paper-based resources into our personal computing environments.

Self-Describing Building Blocks

MERL (Mitsubishi Electronic Research Laboratory): revitalised and modernized Frazer's and Aish's original approach: Architectural Interpretation of 3D Models by using self-describing Lego-like tangible building blocks.
Research on how a computer might assist by automatically identifying architectural elements in a 3D model of a building, and rendering those elements in various styles, with decorative interpretations.

Some other researchers also picked up on Frazers ideas and have started to advance it and apply it to new areas.
Active Cube a system of connecting, self-describing cubes. The user constructs 3-D structures in a virtual environment by simply combining the cubes. A computer recognizes the constructed 3-D structure in real-time. A response to the interaction is also shown by displays/actuators installed on the cube. Application areas are seen in: evaluating 3D spatial constructive ability, educational experiences for children, edutainment, new types of toys, training to increase skills and speed in assembling objects.
Researchers in this group: Benjamin Watson's on interfaces and technologies that will improve highly spatial applications of  interactive computer graphics. former member Ehud Sharlin 

Mechatronic Design Modelling and Simulation System: John Frazer and his team revived and improved the original modules of
The Universal Constructor (originally developed by professor John Frazer and his research team at the Architectural Association in London around 1990) as a new programmable virtual-physical interface .

Tangible, Interactive Toys

Narrative Toys Group at Interactive Institute: Psst! the Programmable SoundScape Toy, the Communicating Dolls, BeastyBox Concept, Student Project Totemette (in Swedish),

Masanori Sugimotos Interaction Technology Laboratory (unfortunately mostly in Japanese...) uses a sensing gameboard for an educational game in environmental issues for children and also for augmented boardgames.

MIT's Lifelong Kindergarden Group: Programmable Bricks, Beyond Black BoxesToys to think with (e.g programmable beads)
Gesture & Narrative Language Group:   AnimalBlocks Embedding story telling functionality into toys:  Sage, Rosebud, SAM, Storymat
Synthetic Characters Group: the Swamped Project - Using Plush Toys to Direct Autonomous Animated Characters

Subproject of EQUATOR, the Hunting of the Snark.

His Master's Voice (in German): semi-autononous ball-robots that react to audio and are used as tokens within a board-game

CACHET (Sterling & Sussex) exploratory study to investigate communication and interactions between children and 'smart' toys, finished late 2002

iToys: Interactive Toys project at IC Cave, Dundee (Scotland): research by Eiman Kanjo

Studierstube Tangible AR for Games - (Master thesis, TU Vienna) - using a glass table and marker-based optical tracking for a ball-game (shooting balloons with a catapult)

Electric Blocks - A Tangible Programming System with Lego-like (Duplo) bricks. Seeing, hearing, touching blocks can detect light, sound and touch. Action blocks emit sound, light or move (a car). Additionaly there are logic blocks which connect sensor blocks and action blocks

Computer Augmented  tabletop games (Ambiente sub-project at Fraunhofer) STARS platform enables easys definition of new games (board games, role play games). Image recognition of "pawns" (play figurines) and illuminated table.

Music Applications   (have become very popular)
(TU Vienna student David Laister has collected most of these links for a presentation in the course on Tangible Computing)

Audio d-touch is a set of 3 applications for music composition and performance: the Augmented Stave, the Tangible Drum Machine and the Physical Sequencer. d-touch is  framework for the development of tangible user interfaces based on computer vision. Work at the Media Lab Europe

The ReacTable from the music technology group at the University Pompeu Fabra in Barcelona, Spain. An electronic music instrument with a tangible user interface; it allows expressive collaborative live performances. The reacTable* is a table-based instrument; it can be played by manipulating a set of objects that are distributed on top of the table surface.

Audiopad at MITs Tangible Media Group, Cambridge

Block Jam (exhibited at Ars Electronica Museum in Linz) consists of a set of electronic cubes which connect and communicate with each other. By arranging the blocks musical phrases and sequences are created, allowing multiple users. The system uses both graphical and tangible user interfaces. Each block has a visual display and a combination of a gestural input and a click-able input.
Augmented Groove  uses augmented reality, 3D interfaces, and physical, tangible interface technologies to allow performers to conduct multimedia musical performances. The performer modulates and mixes compositions by manipulating real LP records.
(both Sony Interaction Lab, Japan)  

The Audio-Cubes from Brussels, a collection of plastic cubes sitting on a performance table, being moved around, stacked, and re-grouped, as the music performance progresses. Each AudioCube sends and receives audio through its faces

Jamodrum (related to CircleMaze exhibited in the Ars Electronica museum in Linz) uses a turntable that incorporates an optical encoder that records the direction players turn the disks. Its a multi-user interactive musical device. This work began at Interval and was developed at Carnegie Mellon's Entertainment Technology Center.  More info on Jam-o-World.

Small_fish as well is exhibited in the Ars Electronica Museum. By moving reflecting objects on a low table, a (virtual) fish swimming in-between these video-augmented objects comes on different tracks. When the fish bounces off the objects or the table edges different sounds are emitted, producing a musical environment. The system can be interpreted both as interactive artwork and interactive music device. The project was inspired by Kandinsky's thinking about connections between music and paintings.

in Amsterdam - Center for Research & Development of instruments & tools for performers in the electronic performance arts, Laboratory, Workshop, ..... Support home of the pioneers of Touch and creators of personal and extremely physical musical instruments.

- lightsensors in dress-up garments used by children create sound and music. An exploration by Kristina Anderson from STEIM.

Design and Interaction Design, esp. student projects

Some design projects by industrial designers or interaction designers might be considered as kind of tangible interfaces. For designers they are simply tangible appliances and products, which now also have some behavior. The relation of form and function has been designers turf all along...

The graduate program at the Interaction Design Institute Ivrea, some student projects: eg. Pouring LightMemory Box, (overview of  student exhibition)

At the Royal College of Arts, London, the 2002 exhibition of the Interaction Design Students on 'Computer Related Design' showed several projects related to tangibles. Have a look at the Body Scanner, Frigo Frigo (non-working link) , and others.

- ID Studiolab, Delft,  is a group of designers, researchers, and students in the department of Industrial Design. The challenge of the ID-StudioLab is to shape the conditions for a satisfactory product experience, thus focusing on experience and emotions.

- the Mads Clausen Institute on Product Development in Southern Denmark and its study program on IT product design   (with focus on tangible IT), (Prof. Jacob Buur) & Tangible User Interaction project

J.P. (Tom) Djajadiningrat does research in Design on tangible interaction, formerly Delft, now Eindhoven and cooperates with Mads Clausen Institute
 his publications

- Malmö University, school of Arts and Communication K3, teaches and researches on Interaction Design, Media- and Cultural studies, Art and Technology. It consists of several studios (space and virtuality, interactive narratives, and  creative environment) some of which are part of the nearby InteractiveInstitute.

- Kunsthochschule für medien köln (Art university for media, Cologne), student projects in the lab3 interface labor kmw are very diverse, falling more into the category of art than into design. Current projects (with text only in German and little online information:  Soundslam (sensory boxing bag),  and "loose space"      
(Kippkick (table soccer that moves), Teppichtasten (carpet with button hot spots to control a video game) - outdated links :-<
PainStation is a custom-made two player table console with a computer game (the classic Pong) that allows to inflict real pain on ones opponent. Players stand on opposite sites of the table. The left-hand is positioned on a sensor field, the right hand plays the game.  Missing the ball is not only annoying, it is also painful, as heat, punches and electroshocks are given to the left hand. The game runs as long as the left hand is kept on the sensor field. Article in Wired

A dupliance is a device that serves two purposes as a two-in-one appliance. It encompasses an information-related activity as well as supports a particular physical activity.  two prototype dupliances; the SkipRope++ and the YoyoPager. Project by Daniel Fällman at Umeå University Institute of Design

Philipps Design is involved in several projects which use Mixed Reality, Ubiquitous Computing and Tangible Interaction concepts:
- Living Memory (LiMe) am EC-Project aimed to enliven Community communication. The LiMe prototype comprises several community nodes forming a distributed network . It also provides tokens with which to store, transport and share information. The nodes  currently consist of a café table, a kiosk, a public display, a desktop, a mobile device and a ‘puck’, allowing users  to browse, view, create and store local community content.
- POGO Active tools for a virtual story world: intertwining physical and virtual media in the process of inventing stories.
- MiMe Multiple Intimate Media Environments
- Nebula:  an interactive projection system to enrich the experience of going to bed, sleeping and waking up. It provides intuitive and natural ways of physically participating in a virtual experience, through simple body movements and gestures. A ceiling projector is linked to a database of content. Once users have selected the content for  projection, they can manipulate it by adjusting their sleeping positions. Content is selected by placing a smart 'pebble' into the bedside pocket.

The Placebo Project by Dunne & Raby

Time's Up - Laboratory for the construction of experimental situations. Art group/project situated in Linz, Austria. Doing situationist ?research? around issues of control, perception and biomechanics. Events/installations like Sensory Circus / Public Balance Experiences  

Mixed Reality, Augmented Reality and Tangible Interaction

The EQUATOR project, is an  Interdisciplinary Research Collaboration(IRC) supported by EPSRC that focuses on the integration of physical and digital interaction. It investigates into Mixed Reality for performances, use in domestic environments, enhancing community care, also in Wearables, Augmented Reality and other avenues of integrating physcial and digital worlds.
The Hunting of the Snark is a tangible interactive playgood for schildren.

Pin&Play Project: a new approach of ad-hoc networking among objects that people can attach to large surfaces, using up to now (smart) pins and wall switches

Shape project (Situating Hybrid Assemblies in Public Environments) devoted to understanding, developing and evaluating room-sized assemblies of hybrid, mixed reality artefacts in public places combining interactive visual, sonic and physical manipulable devices. The Storytent (pointing with flashlights at tent walls makes things visible), Ghostship exhibit (interactive art installation). Part of EU Disappearing Computer inniative

BUILD-IT picked up on the original Bricks-idea. Physical blocks are used as tangible handles to interact with a projected model. Handles are temporarily connected and frequently changed to choose tools, select objects and manipulate them. Thus the system can be categorized as Augmented Environment with some ideas of Tangibles. The application domain is factory planning.
Matthias Rauterberg (ETH-Zürich/Uni Eindhoven) & Morten Fjeld (researched choice of physical handle form and navigation methods): BUILD-IT http://www.iha.bepr.ethz.ch/pages/forschung/MMI/Projects/bit/bit.htm

PLAY Research Group and Future Applications Lab at Victoria Interactive Institute (used to be one group) also has related projects, this is not really tangible interfaces, but uses some related ideas  (Token-Based Acess to Digital Info & Enhancement of Everyday objects): Projects on  Smart_ITs, Amplified Reality

Teco  Telecooperation Office, University of Karlsruhe : MediaCup & Smart_ITs Project (together with PLAY)
mostly Ubiqoutous Computing oriented, now does a project  with PLAY research group: " a  vision of computation embedded in the world"; "Smart-Its" - "small-scale embedded devices that can be attached to everyday objects to augment them with sensing, perception,  computation, and communication, also cooperating with Lancaster
See also smart-its.org   and the RELATE project: "Assessment of relative positioning technologies for compositional tangible interfaces"

Proactive Furniture Assembly at the ETH Zürich - help for assembly by attaching sensors and lights to the furniture parts

Tiles  A Mixed Reality Interface, where interactive virtual objects are overlaid on the physical environment, that allows users to dynamically add, remove, copy, duplicate and annotate virtual objects anywhere in the 3D physical workspace. Paper cards with visual tracking patterns are attached to a whiteboard. The tiles are detected and augmented (seen through a semi-transparent HeadMounted Display).   (web page does not seem to exist any more....)
ZTiles (new Tiles version) at the Interaction Design Centre  at the University of Limerick in cooperation with the MIT Media Lab Europe in Dublin. (also this page and the MIT Europe site)

PLANTILES interactive stage consisting of fifteen wood tiles in a 3 by 5 layout with a projected display output. Depending on where the user stands and how the user moves on the stage, corresponding movies will be displayed on the output screen. embeds computational meaning in a spatial relationship (termed by authors as TUI also not being really tangible or graspable....)

CAMELOT: Project group on Augmented Reality at GMD/FHG in Bonn, using physical props for Mixed Reality: ARTHUR (Augmented Round Table for Urban Planning), Mixed Reality Stage (testing lighting shows), augmentation is visible using HMDs.

The Ambiente Project of GMD in Darmstadt develops "RoomWare": computer-augmented workspaces and furniture (interactive tables, whiteboards....) which are connected in one network. Although the project realizes mostly visions from Ubiquitous Computing & Augmented Environments, it includes some ideas of Tangibles (e.g. the passenger to move information or work objects in-between workspaces)

Bremen Student Project METHEA (Bachelors in Media Informatics) under supervision of members of artec, showed their results as an interactive Mixed Reality Installation: Sensoric Garden (text on web site in German) - at 3 evenings in June 2001, visitors came to the public garden in Bremen (the old theatre hill) for a post-modern interactive promenade. 7 installations invited to look and interact 
more photos (got lost due to server crash, need to re-launch this...)

artdecom project at the University of Lübeck, Germany works with schools, experimenting with Mixed Reality. Dragon's World, an interactive music show or performance produced by a primary (elementary) school class. A learning project across the subjects of technology, art, dance and handicraft, developed a story which was transformed into a Mixed Reality performance. Moving on the stage activated certain "hot spots" and produced sound and images belonging to the story.

»Cyberpunch« Mixed Reality experimental Puppetery (Berlin, Germany): the puppets move from stage to the screen and into virtual space. Virtual and real puppets interact with other. Kasper and Gretel play "Punch and Judy" every day, until Kasper makes a pact with the devil to escape routine and is taken into the world of cyberspace.

Whisper : wearable body architecture. An art project on wearable computing and human senses. The group is developing technology and communication metaphors that enable networked wearable devices to communicate affective states expressively in a continuous manner. whisper[s] are wearable body architectures that read physiological data and transmit this information through graphics, sounds or haptics. The aesthetic is deliberately ambiguous.

 whisper[s] are wearable body architectures that read physiological data and transmit this information through graphics, sounds or haptics. Our aesthetic is deliberately ambiguous,

The Automatic Pooltrainer from Aalborg, developed in the context of graduate student projects and master theses.
actibits, a collection of interactive exhibits (using eg. infrared/electronic batons) designed by Jan Borchers - I should go and try the Virtual Conductor at the House of Music Museum in Vienna.....

XFR - Experiments in the Future of Reading. A Xerox Parc project with an exhibition design and lots of exhibits showing possible/impossible visions of how reading might change. Only website I could find on this was in Parcs pressroom, due to the dissolvement of Xerox Parc into Parc.

Studierstube Augmented Reality Project - bunch of projects based on software platform Studierstube developed at Vienna University of Technology

Tangible Hypermedia (Using the AR Toolkit)

Tangible Interfaces for Volume Navigation using 3D interaction props and optical markers at the TU Eindhoven

TUISTER:  a multi purpose TUI device for two-handed interaction with combined input and output capabilities.

The Planar Manipulator Display - using movable objects on a TUI (with built-in motors)

Tabletop systems & research

MERL's DiamondTouch system can detect multiple, simultaneous touch events, but also  identify who is touching where. The interaction surface is filled with arrays of antenna's, each sending a unique signal. Each user has their own receiver, generally attached to their chair. When a person touches the surface, energy from nearby antennas is coupled through the user to their receiver. This is how the system determines who is touching where. DiamondTouch is said to have a tangibles option !
see also DiamondSpin

Stanfords iTable (computer-based tabletop display for collaboration)

Tabletop research at Edgelab, Canada: organised some workshops on co-located tabletop collaboration

The Pond, multi user tabletop display used for information browsing and visualizing, using flocking behavior to imply groupings. Video recognition of gestures, pressure pads and tag input. RFID tags designate search categories or are used to mark data snippets

Technology Stuff


Phidgets™, or physical widgets, are software building blocks that help a developer construct physical user interfaces. Phidgets abstract and package hardware devices like sensors, motors, power outlets, etc., thus making programming and controlling physical interface easier.
Now derived into a company and electronic store, that produces and sells  Phidgets™ --  electronic modules for controlling and sensing the "real world" from your computer.  The Phidgets are USB devices, and use a free software library to interface to your application or development environment! 

Papier-Mâché: Toolkit support for tangible input from Group for User Interface Research, University of California (see Designers' Outpost)
open-source toolkit for building tangible interfaces using computer vision, electronic tags, and barcodes using a high-level event model supporting portability across tracking/registration technology

iStuff: toolkit of physical devices, and a flexible software infrastructure for rapid prototyping. Software framework includes a configurable "Patch Panel" intermediary to simplify the dynamic mapping of devices to applications. Some documentation

Other technical stuff

a student report on RFID technology (no promise that everything is correctly described!)

The hardware (sensing board, based on RFID) inside the PitA-Board (L3D, University of Colorado) comes from DGT , a dutch company producing electronic chess products. The company is willing to sell the inner hardware without the chessboard cover. Each field has a 2 inches square size. Due to the chessboard-inheritance, the circuitry comes as 8x8 array, but boards can be arranged in tandem. If the market evolves, the company may be willing to develop smaller circuitry, providing smaller sensor fields and thus higher resolution.

circuit bending - a handbook on how to re-wire audio or toy games: "Circuit-bending is an electronic art which implements  creative audio short-circuiting. This renegade path of electrons represents a catalytic force capable of exploding new experimental musical forms  forward at a velocity previously unknown. Anyone at all can do it; no  prior knowledge of electronics is needed. "

The HIT Lab at University of Washington is making available the Augmented Reality tracking libraries (ARTookKit and MagicBook) used as the basis for the Shared Space project and SIGGRAPH demo. The libraries use computer vision techniques to precisely calculate a camera position and orientation relative to a tracking marker. The programmer can then use this information to draw 3D virtual objects that are exactly aligned with a real object.

Z-Tiles pressure sensitive pads, (site disappeared ???) which respond to weight changes (= movement) and can be used as surface for floors to make them interactive, developed at Interaction Design Centre Limerick and Media Lab Europe (one needs Shockwave to see some of the content)

PLANTILES interactive stage documentation which might enable building similar pressure sensitive floor pads  

Tactex produced a technology from fiber optic based pressure sensing material that detects pressure everywhere on its surface

DDR (Dance Dance Revolution), a new game (machine)  in video arcades, is a dancing simulation game that you play with your feet.  People dance to a given song and step on certain spots on the platform whenever an arrow blinks. Hits or misses affect the life score, just as in other video games. The game is done as a performance (in public, trying to make dancing look good) or dance collaboratively. Some links AboutDDR, on the Game idea, and I even found a do-it-yourself guide for making such a platform

AnotoPen (by Anoto)- a pen which, when writing on paper with printed micro-patterns, registers the writing via small cameras inside the pen and sends it to the computer network. Enables having the real handwritten document while providing a digital copy. But one needs to buy the special paper.

Vernier LabPro(R)  can be used for teaching and experimenting with sensor technology. Its a small device which plugs into any computer to transfer data previously (or online) collected via a range of different sensors, which can be bought as well and plugged into the device (Plug & Play...). There are sensors for temperature, motion detectors, accelerometers, gas pressure, humidity, light, EKG .....

Mimio - simply pens enhanced with signal givers and a whiteboard or flipchart surrounded by two beams detecting the signals - gives a digital copy of physical writing

FingerWorks.com Zero-Force keyboards and pointing devices with MultiTouch gesture input.

Soundbeam "- a distance-to-voltage-to-Midi device which converts physical movements into sound by using  information derived from interruptions of a stream of ultrasonic pulses." -- see also the CARESS project (EC-funded) on interactive acoustic environments for school kids
or "The ultimate MIDI controller" by Infusion

EnOcean GmbH sells maintenance-free, battery-less and cable-less switches and sensors. They enable radio transmission without batteries or external power supply. Radio switches and sensors “supply themselves” by available environmental energy. Final products are for example light switches that emit a radio signal when being pressed. This signal contains a unique identification number, is registered and identified and lights the lamp.
Sounds like something that could be useful in TUI-contexts!

Toys that sound useful for "cannibalizing": Neurosmith company

pressure sensing material from Tactex

Hardware-Technology and Software Links taken from Brygg Ullmer's PhD thesis (RFID systems, MIDI etc. )

Bibliographies & other resources

short bibliography by the Digital Mouse House group

Brygg Ullmer`s course on Giving Physical Form to Digital Information, Fall 2002 at Hong Kong Polytechnic University, School of Design

Online Proceedings of Workshop Physical Interaction - workshop on Real World User Interfaces at Mobile HCI 03 in Udine (italy)

Report of Workshop about TUIs: Eva Hornecker, Lone Malmborg, Hal Eden. Designing Tangible User Interfaces to Support Participation - Report of a PDC 2002 Workshop, 17 pages. artec-paper Nr. 97, Oktober 2002. (technical report) pdf

(there are more projects and reasearchers, but not all of them have web pages or I might have missed some. Send me an E-Mail if you have any other resources fitting into this list to include:
eva  [at]
ehornecker [dot] de  

Legal Disclaimer

Eva Hornecker                                             content last updated October 2004                                         Go to My Homepage