MIT Ultrasound Wristband Turns Human Hand Movements into Real-Time Robotic Control

The human hand is the body's most agile instrument, coordinating 34 muscles, 27 joints, and more than 100 tendons and ligaments to execute a vast range of delicate movements and gestures. Replicating that dexterity in robots has long been a formidable challenge, partly because researchers have struggled to precisely capture what is happening beneath the skin. A team at the Massachusetts Institute of Technology (MIT) is now pioneering a promising new approach.

An Ultrasound Wristband That Sees Through Skin

MIT Professor of Mechanical Engineering Xuanhe Zhao, working with colleagues at MIT and the University of Southern California, has designed a wristband embedded with ultrasound patches — miniaturized versions of the transducers used in clinical ultrasound equipment, bonded to the skin via a biocompatible hydrogel adhesive. As the wearer moves their hand, the device generates continuous ultrasound imagery of the wrist's muscles, tendons, and ligaments. An AI algorithm trained on meticulously annotated ultrasound images then translates that feed into the corresponding positions of all five fingers and the palm.

Co-first author Gengxi Lu, a former MIT postdoctoral researcher, offered a vivid analogy to explain the concept: "The tendons and muscles in your wrist are like the strings of a marionette, and your fingers are the puppets. Every time you image those strings, you know the state of the hand."

Wireless Robotic Control: Piano Playing and Basketball Shots

In live demonstrations, the research team showed how a wristband wearer could wirelessly control a robotic arm — with the mechanical hand mirroring the user's gestures and pointing movements in sync. Operating in what the team describes as a wireless marionette-style interaction, wearers were able to guide the robotic hand to play simple melodies on a piano keyboard and shoot a miniature basketball into a small tabletop hoop. The same wristband was also used to manipulate objects on a computer screen, for example using a pinch gesture to zoom a virtual object in or out.

Road Ahead: Humanoid Robot Training and VR Applications

The researchers plan to shrink the wristband's hardware — currently comparable in size to a smartphone — and aim to enrich the AI model by incorporating motion data from volunteers with a wider variety of hand sizes and finger shapes.

The team envisions building a large-scale hand-motion dataset that could be used to train humanoid robots to perform precision tasks such as surgery, and to allow users to naturally grasp, manipulate, and interact with objects inside design applications, video games, or other virtual environments.

"We believe that tracking dexterous hand movements through wrist-worn imaging represents the state of the art," said Professor Zhao. "We think these wearable ultrasound wristbands can provide an intuitive and versatile means of control for both virtual reality and robotic hands."

The team's ultimate goal is a wearable hand tracker accessible to anyone, enabling real-time, high-dexterity wireless control of humanoid robots or virtual objects.

原文來源： 查看原文