Framework

History

The rise of wearable technology

Caroline Bonaparte wearing a wristwatch (Barnebys, 2023)

The rise of wearable technology began when aristocratic women first commissioned wristwatches, bracelets with watches and ring watches. Caroline Bonaparte is pictured wearing a wristwatch at a time (1810–1812) when aristocratic men still preferred pocket watches. In the military and in sports, men eventually began to wear wristwatches (Barnebys, 2023). In 2015, Apple released the first generation of its smartwatch. The name Apple Watch refers to the tradition of wearing timepieces, which helped establish a category of devices that can do much more than tell the time at a glance.

The three pillars of mixed reality

Watches have introduced temporal registration as the first pillar of mixed reality. Smartwatches include spatial registration, the second pillar. They are capable of sensing location and movement and overlaying contextual information that is interactive in real-time. Currently, human augmentation is shifting from the first wave, "Overlay", to the second wave, "Entryway" (Papagiannis, 2017, p. 5). A third pillar of mixed reality seems to be manifesting itself: The personalisation of context.

Current Shifts

The Great Adventure of Material World (Lu Yang, 2020)

Towards Augmented Humans

Humans have been shifting from distinct entities towards augmented humans (Papagiannis, 2017). This transformation manifests in wearable technology, that acts as prostheses. Humans relate through this pervasive layer of their existence as hybrids and socio-technical beings. This shift can be traced back through anticipatory art or popular culture references, such as cyborgs. The Surrealist movement provided a foreshadowing moment, which resonates with contemporary digital art. “Parreno’s works (...) liberate the viewer from the common belief that the world is neatly divided in two spheres: us humans and things out there. Time to rethink” (Birnbaum, 2018, p. 21). Subject and object are merging into one.

Towards Blended Reality

Blended reality increasingly permeates everything we do. Our “attention and other cognitive resources are often split between dealing with the physical world in which we are located and interacting with the digital (...). While we are physically in one place, we are often mentally in another, or more than one place.” (Waterworth & Hoshi, 2016, pp. 3–4). It is challenging to integrate presence across different realities and blend them harmoniously.

Towards Entryway Applications

Mobile mixed reality apps have been around for many years: Snapchat, Instagram or Tiktok are good examples of AR apps that follow the “Overlay” wave. These applications offer virtual filters that superimpose physical reality. What makes them mixed reality applications is spatial registration in combination with real-time interactivity. Often these apps depend on a camera and a screen, and demand a lot of attention. Calm technology can serve as a source of inspiration for the development of MR/AR applications that require less attention and blend more harmoniously into physical reality.

“We are entering a second wave of AR, which I call “Entryway”, creating a more immersive, integrated and interactive experience. (...) You are the context that defines the experience” (Papagiannis, 2017, p. 5).

Framework – Design for Extended Reality

Layered Augmentation (adaptation of the HCD triade human–product–context) & Extended Reality continuum (Milgram, Takemura, Utsumi, Kishino, 1995)

A different take on More-Than Human Design

Human Centred Design reaches its limits when dealing with complex software. “From a perspective of dealing with complexity, human-centered design (...) is conceptually grounded in the relationship between a person and a tool” (Giaccardi & Redström, 2020). ‘More-Than Human Design’ is an emerging framework that indicates how to constructively deal with today’s complex realities. I am approaching the ‘more’ in ‘More-Than-Human Design’ with concepts from Design for Extended Reality to shape the relations and dynamics of human augmentation through wearable technology.

Extended Reality

Extended Reality can be thought of as a continuum that spans across a spectrum (Milgram, Takemura, Utsumi, Kishino, 1995). On the left side is the real environment and on the right side is the virtual environment. Digital twins originate in either either side, from where they augment reality towards the other side. An AirTag can give any physical object a virtual presence.

Mixed Reality

I define mixed reality as a medium in which “real” and “virtual” things or surroundings blend. They are spatially registered and interactive in real-time. This definition shares the key characteristics with Ronald T. Azuma’s definition of Augmented Reality (Azuma, 1997).

Augmented Human

In this project I am interested in two particular aspects of human augmentation: Treating the body as an interface and registering the layers of analog and digital means of augmentation. Embodied behaviour can be treated as meaningful input for mediated interactions. With layered augmentation, visualised as ripples, I intend to capture the things that embrace and surround a person, that enhance their ability to self-actualise or relate to things, other beings and surroundings. These layers can feel like a part of themselves: The clothes they wear, the sun glasses, the earbuds. These layers can act as shields, membranes or prostheses. Instead of speaking indifferently of humans, I distinguish a walking human from a biking, or skating human.

Models

Layered Augmentation & Reality-Virtuality Continuum are two models that illustrate the elements of human augmentation and the immersion along the reality-virtuality continuum.

Ananyo is cycling home while Christina is sitting on the couch. Christina picks up her phone to text him.

The OS of his smartwatch registers Ananyo’s situation and makes Christina aware that he is cycling with his smartwatch and earbuds on.

MTHD Glossary

I realised the need to find a vocabulary that represents the believes, relations and dynamics when I talk about interaction design.

This glossary is an attempt to clarify the role of concepts in each design framework. I added concepts from Design for Extended Reality to each side.