New app allows your smartphone to understand gestures data

Researchers have developed a first of its kind app that enables users to operate their smartphone using gestures. The app uses the smartphone’s built-in camera to register its environment.

By mimicking the firing of a pistol, for example, a user can switch to another browser tab, change the map’s view from satellite to standard, or shoot down enemy planes in a game. Spreading out fingers magnifies a section of a map or scrolls the page of a book forward.

The information that the app registers – the shape of the gesture, the parts of the hand – is reduced to a simple outline that is classified according to stored gestures.

44749497.cmsThe app, developed by Jie Song, a master’s student in the working group headed by by Otmar Hilliges, Professor of Computer Science at ETH Zurich, then executes the command associated with the gesture it observes. It also recognises the hand’s distance from the camera and warns the user when the hand is either too close or too far away.
“Many movement-recognition programmes need plenty of processor and memory power,” said Hilliges, adding that their new algorithm uses a far smaller portion of computer memory and is thus ideal for smartphones.

The app’s minimal processing footprint means it could also run on smart watches or in augmented-reality glasses. The app currently recognises six different gestures and executes their corresponding commands.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s