Interacting with Levitating Objects


Interacting with Levitating Objects

One of our goals in the Levitate project is to create new types of display using levitating objects.

These new displays use physical objects to represent information or to create user interface elements, rather than pixels on a flat solid screen. There are many unique properties of this type of display; for example, their display elements can be moved in three dimensions and people can see through the display and its content, to people or objects on the other side. The following video gives an example of this:


With these novel capabilities, designers can create new interactive experiences. We have been working with artists and designers to explore the types of user interface and experience that can be created using levitating objects. This work is ongoing and we're excited to be able to share more about this in a future blog post.

Displays of levitating objects also offer new challenges for interaction designers. There are lots of established ways of interacting with ordinary displays; like touching buttons on a screen or pointing at icons using a mouse or trackpad.

What are the equivalent interactions for levitating objects?

People will not always be able to reach out and touch a levitating object and controlling a levitating 'pointer' with a mouse or trackpad might be quite difficult, because these two-dimensional input devices do not have a direct mapping to the 3D physical world.

We have therefore been looking at new ways of interacting with levitating objects, based on hand gestures in mid-air. One of these gestures lets users target a levitating object by pointing their finger at it. This is much more straightforward than using a mouse or trackpad to point at objects in 3D. Another interaction challenge is giving users feedback, to let them know what is happening while they interact with a levitating object. We created a simple way of giving feedback, based on object movement. As the following video shows, we shake an object when someone points at it:


This work [1] is the first step towards fully interactive displays based on object levitation. Selecting objects by pointing at them allows more complex operations to be performed. Users might next perform another gesture to move the objects or change their appearance. For example, the video at the start of this page shows a rotating levitating cube, which could be directly controlled in real-time by a user moving their hands in mid-air.


Written by Dr Euan Freeman



[1]  Point-and-Shake: Selecting from Levitating Object Displays - Euan Freeman, Julie Williamson, Sriram Subramanian ,and Stephen Brewster - CHI 2018