Interacting with Levitating Objects

 

Interacting with Levitating Objects

One of our goals in the Levitate project is to create new types of displays using levitating objects.

These new displays use physical objects to represent information or to create user interface elements, rather than pixels on a flat screen. There are many unique properties of this type of display; for example, their display elements can be moved in three dimensions and people can see through the display, to people or objects on the other side. The following video gives an example of this:

 
 
 
 

With these novel capabilities, designers can create new interactive experiences. We have been working with artists and designers to explore the types of user interface and experience that can be created using levitating objects. This work is ongoing and we're excited to be able to share more about this in a future blog post.

Displays of levitating objects also offer new challenges for interaction designers. There are lots of established ways of interacting with ordinary displays; like touching buttons on a screen or pointing at icons using a mouse or trackpad.

What are the equivalent interactions for levitating objects?

People will not always be able to reach out and touch a levitating object and controlling a levitating 'pointer' with a mouse or trackpad might be quite difficult.

We have therefore been looking at new ways of interacting with levitating objects, based on hand gestures in mid-air. One of these gestures lets users target a levitating object by pointing their finger at it. This is much more straightforward than using a mouse or trackpad to point at objects in 3D. Another interaction challenge is giving users feedback, to let them know what is happening while they interact with a levitating object. We created a simple way of giving feedback, based on object movement. As the following video shows, we shake an object when someone points at it:

 
 
 
 

This work [1] is the first step towards fully interactive displays based on object levitation. Selecting objects by pointing at them allows more complex operations to be performed. Users might next perform another gesture to move the objects or change their appearance. For example, the next video shows a group of objects being rotated, which could be controlled in real-time by rotating the hand:

 
 
 
 

Written by Dr Euan Freeman

 

References: 

[1]  Point-and-Shake: Selecting from Levitating Object Displays - Euan Freeman, Julie Williamson, Sriram Subramanian ,and Stephen Brewster - CHI 2018