The Tangible Interaction Framework

For the new Tangible Interaction Card Game (brainstorming exercise) presented at TEI'2010 please select 'T.I. Card Game' on the menu on the left!

‘Tangible Interaction’
denotes systems that rely on
- tangibility / materiality
- bodily / embodied interaction
- physical representation of data
- embeddedness in real space, and augmentation of physical spaces.

In different systems these characteristics can be of various intensity. Tangible Interaction is an umbrella term, which encompasses approaches from HCI, computer science, product design and the interactive arts.

Research questions the framework contributes to:

- What is the user experience of tangible interaction?
- Why does tangible interaction work so well for users?  (Paul Dourish asks this)
- How can we conceptualize the social aspects of tangible interaction?
- Which concepts can we employ and what perspectives take for analysing and evaluating systems?
- Are there concepts that can guide or inspire the design of new systems? 


Framework Themes and Concepts

The coloured boxes represent themes on tangible interaction. These are not mutually exclusive, but interrelated, offering different perspectives on tangible interaction that can guide our thinking and sensitize for specific questions. Each theme is elaborated by a set of concepts, such as e.g. Inhabited Space or Embodied Constraints. Concepts are further elaborated with sensitizing questions.

Tangible Manipulation refers to the material representations with distinct tactile qualities, which are typically physically manipulated in tangible interaction.
In tangible interaction we interact very directly with objects and experience their physicality. We aim for lightweight interaction that enables us to experiment and have a dialogue with the objects we interact with. This is not always the same as intuitiveness - only the basic level of interaction needs to be intuitive by providing a legible mapping of human action and effect, the semantics of use may be highly complex - and need to be for complex domains...

In particular we should attempt to go beyond 'simple mappings' - tangibles that scale might be the big design challenge...

Spatial Interaction refers to the fact that tangible interaction is embedded in real space and interaction therefore occurrs by movement in space.
We humans are spatial beings by nature. This means that our body is a reference point for perception and that spatial qualities are always unconcsiously interpreted and acquire psychological meaning. We move around in space and move objects around, trying to configure space according to our needs. Spatial interaction also allows us, by moving in it, to make use of our entire body, it enables observation and thus implicit awareenss as well as explicit performance...

Embodied Facilitation highlights how the configuration of material objects and space affects and directs emerging group behavior. The analogy here is with group facilitation methods.
The set-up of physical space, configuration and availability of interaction objects as well as the logical space provided from the digital software provide structure, constraints, and options for what we can do and how we behave. They can subtly influence social interactions by allowing, directing, suggesting or limiting certain activities. They can make it difficult or easy to observe and understand what is going on and to become active oneself. The choice of representations also affects whether users can relate to them, 'read' them, and feel invited to interact.

Expressive Representation focuses on the material and digital representations employed by tangible interaction systems, their expressiveness and legibility.
Hybrid representations combine tangible and digital elements of representation. Representations do communicate to us - they have expression. We 'read' them, interpret them, and manipulate them.  Tangible interaction requires us to design both digital and tangible elements, to make them expressive and legible, and to combine them in a 'good' way so they augment each other and their relationship is legible for users. Which representations we choose as designers affects how users can employ them in externalizing their thinking and as communication prop.

Using the Framework

The function of a framework is to focus our view, providing us with concepts that systematize our thinking and allow for reflection. In this sense this framework does not intend to be all-encompassing or to provide rules. It is hoped that it can assist both in analysis and design, by sensitizing and orienting to relevant issues.

Sensitizing Questions
The concepts are elaborated into sensitizing questions. These are colloquially phrased and (hopefully) can be understood without reading any of the papers on the framework. They should be used as inspiration, provocation, or suggestion, triggering discussion and ideas. Each question is printed on an index card, providing freedom in how to work through the entire set of questions. 

Example questions:
- Can users grab, feel and move ‘the important stuff?
- Are there powerful (non-literal) representations that transform the problem?
- Can you use your whole body?
- Can everybody see and follow what's happening?
- Can all users get their hands on the central objects of interest?
- Are representations legible, meangingful, and expressive?

When and how are themes, concepts and questions relevant?

Some of the themes are more relevant to tangible interaction in a strict sense, and others are also useful for thinking about other interaction styles. Tangible Manipulation will be most relevant if there exist physical input elements, while Spatial Interaction also seems useful to think about e.g. mobile devices and Ubiquitious Computing.

Some concepts and questions may be completely irrelevant for a particular case - or the design goal might actually be the opposite (e.g. we might want to design invisible actions in order to preserve privacy, or to restrict access to the 'red button' to one person only).

Origins of the framework

The framework has evolved out of my PhD research on the role of tangible interfaces for collaborative interaction, synthesizing insights from this project, findings collated from literature study, and ideas and insights gained from more recent projects. It  integrates and fuses relevant recurrent themes and concepts from previous attempts at conceptualizing tangible interaction. The main structure of the framework has been developed with the mentoring help of Jacob Buur during a research visit at the University of Southern Denmark.

Over the past years the framework has been revised and refined - this concerns e.g. the phrasing of questions, refined explanation of concepts, the structure of the framework (getting rid of overlaps), its use in analysis of case studies, collating of further illustrating examples, employment of the framework in design, etc.

'The Tangible Interaction Framework Card Game'

One of the aims of this research has been to make the theory (the framework) practically useful, supporting design ideation or the discussion of potential evaluation criteria for a given project.  'For this, the framework has been translated into a set of cards, which each depict one of the framework's 'provocative questions' along with inspiring images. These are then used in a brainstorming exercise that uses a 'game' format. The goal of this is to combine the freedom of a creative idea exploration process with the structure of a guiding framework.

Different from most other creative brainstorming techniques, the triggers are not random, but come from the framework. Different from design guidelines and checklists, there are no lists to go through, no right and wrong answers, only 'things to think about'. The cards randomize the order in which the questions are talked about, but still allow the team to develop their own structure.

The card exercise was presented at TEI'10.

Please visit this page for more details: T.I.Card Brainstorming Game


Central Publications 

E. Hornecker. Creative Idea Exploration within the Structure of a Guiding Framework: The Card Brainstorming Game. Proc. of TEI'10. ACM. 8 pages pdf 

Eva Hornecker, Paul Marshall, Yvonne Rogers (2007): Entry and Access – How Shareability Comes About. Proc. of DPPI’07 (Designing Pleasurable Products and Interfaces). ACM pdf
(This further unpacks some concepts, merging them with ideas from Yvonne. The focus here is on how a system, interface, or device engages a group of collocated, co-present users in shared interactions around the same content. Entry points invite and entice people into engagement, access points enable users to join a group’s activity)

Eva Hornecker and Jacob Buur (2006): Getting a grip on tangible interaction: a framework on physical space and social interaction.   CHI 2006 © ACM
(gives a high-level overview of the entire framework, positions it in existing research, and discusses three case studies in relation to the framework concepts)

Eva Hornecker (2005): A Design Theme for Tangible Interaction: Embodied Facilitation E-CSCW 2005 © Springer
(focuses on Embodied Facilitation, that is how the physical set-up and configuration influence group behavior similarly to the design decisions a human facilitator for a group session might make, explains the concepts along with corresponding guidelines and gives examples from case studies)

additional publications on the framework:

Eva Hornecker (2006): Physicality in Tangible Interaction: Bodies and the world. Position Paper for the First International Workshop on Physicality, Lancaster Universtiy, 6-7 Feb. 2006, UK (organizers: Masitah Ghazali, Devina Ramduny-Ellis, Eva Hornecker, Alan Dix) pdf
(gives a summary of the framework and discusses the role of physicality in respect to the human body and the physical world within the framework -  thus a bit more detail on issues related to physical design and embodied interaction)

Eva Hornecker. Space and Place – Setting the Stage for Social Interaction. Position paper for ECSCW05 workshop 'Settings for Collaboration: the role of place
(this focuses on Spatial Interaction and Embodied Facilitation, putting the themes in context to the discussions about the space-place distinction - which in my view is purely theoretical, as human experience of space is always situated, spatial constraints directly influencing our perception of space and our behaviors - this is stuff for further work on)

further studies / publications related to aspects of the framework

Eva Hornecker. Interactions Around a Contextually Embedded System. Proc. of TEI'10. ACM. 8 pages

Paul Marshall, Rowanne Fleck, Amanda Harris, Jochen Rick, Eva Hornecker, Nicola Yuill, Yvonne Rogers & Sheep Dalton. Fighting for Control: Embodied Negotiation of Access to Digital and Physical Representations. Accepted note at ACM CHI 2009. (4 pages)  (evidently, this has to do with Embodied Facilitation - what kinds of access through which physical actions do different organizations of the interface enable - including conflicts over control)

Eva Hornecker, Paul Marshall, Nick Sheep Dalton, Yvonne Rogers (2008): Collaboration and Interference: Awareness with Mice or Touch Input. ACM CSCW 2008. San Diego. 167-176  (analysing effect of different types of 'access points' on awareness and coordination, following on on the DPPI paper)

Paul Marshall, Eva Hornecker, Richard Morris, Nick Sheep Dalton, Yvonne Rogers (2008): When the fingers do the talking: A Study of Group Participation With Varying Constraints to a Tabletop Interface. Proc. of IEEE Tabletop 2008. pp. 37-44 
 (analysing effect of different types and numbers of 'access points' on equity of participation, following on on the DPPI paper)

Case Studies

These papers provide some more background on the original case studies in the central framework publications:

Eva Hornecker, Matthias Stifter. Learning from Interactive Museum Installations About Interaction Design for Public Settings. Proceedings of OzCHI 2006. Sydney. ACM, 135-142
pdf preprint  (the TMW exhibition evaluation)

Eva Hornecker, F.W. Bruns. Interactive Installations Analysis - Interaction Design of a Sensory Garden Event -.  Paper at IFAC/IFIP/IFORS/IEA Symposium on the Analysis, Design and Evaluation of Human-Machine Systems 2004 (Atlanta, GA). CD-ROM Proceedings Preprint (pdf)  (on a set of installations, one of them the clavier keyboard)

Hal Eden, Eva Hornecker, Eric Scharff. Multilevel Design and Role Play: Experiences in Assessing Support for Neighborhood Participation in Design.  In: Proceedings of  DIS´02. ACM. pp. 387-392   Preprint 

These are further case studies where the framework was used for analysis or to inspire design ideas:

Eva Hornecker. Tangible Interaction Design, Space, and Place. Position paper for 'Space, Place and Experience in HCI' Workshop at Interact 2005
(this focuses on Spatial Interaction and Embodied Facilitation, using the setting of a project I was involved in as imaginary starting point to brainstorm design ideas. We did this with lab collegues as kind of a 'design game'. This was  a thought experiment to see how the framework can relate to an actual real setting and how it could inspire design. It also shows that some framework themes are highly relevant to mobile interaction scenarios! )

Cliff Randell, Eva Hornecker, Eric Harris and Geraldine Fitzpatrick.Device Design for Learning Using Place, Intuitive Behaviour and Collaboration. Position Paper for UbiPhysics workshop at Ubicomp 2005, Tokyo. pdf
(combines description of an Equator project with thinking about possible avenues for devices in this context, using the framework guidelines as inspiration)