The May 2024 issue of IEEE Spectrum is here!
Non-invasive, AI-assisted device converts neck movements to words and phrases flat wire winding coil
A wearable patch that senses a person's neck movements can, with some external AI assistance, speak a range of phrases to provide vocal capabilities for those without.
For many, an inability to speak is a significant issue: A 2014 study of vocal disorders in the U.S. found that nearly 18 million adults had difficulty using their vocal tracts to speak, while over half of that group experienced debilitating speech issues for periods longer than a week.
A new non-invasive wearable device provides one possible way to address this medical need. The technology consists of a lightweight patch adhered to a person’s neck, and that patch measures the person’s neck movements. Then an off-device processor translates those signals into speech, after which that speech audio is played out in lieu of the person’s voice.
“Similar to how the material converts muscle movements into electricity, it can also induce electricity as an input signal into mechanical vibrations in the device that produce sound.” —Jun Chen, UCLA
The research team, led by Jun Chen, assistant professor of biomedical engineering at UCLA, has created a flexible and electromagnetically responsive wearable that measures subtle neck muscle movements. A downstream device—not the sensing patch—then decodes the muscle movements the patch has sensed into speech, using a machine learning algorithm trained to recognize a set dictionary of phrases.
This technology could benefit people with injuries and diseases involving vocal fold paralysis, as well as those who have had surgeries like laryngectomies—removal of the larynx, which contains the vibrating muscles—that involve the removal of some or all of a person’s vocal folds.
The device’s performance is limited, however, by how many sentences it can play back. The present prototype AI model only selected from among five sentences: “Hi, how are you doing today?”; “Hope your experiments are going well!”; “Merry Christmas!”; “I love you!”; and “I don’t trust you.” The model was trained and tested on neck movements measured from people without any speech disability, who were asked to move their necks as though they were speaking but without vocalization.
The new device joins other medical technologies that treat vocal fold disorders, such as prosthetic larynges and voice boxes, which vibrate the neck to recreate lost vocal fold movement. The UCLA patch approaches the problem differently, converting a user’s unvoiced neck muscle activity into computer-generated speech.
Dr. Barbara Messing, a clinical educator for medical device company Atos Medical and unaffiliated with the UCLA project, said the new approach is a welcome addition to the space of assistive speech devices: “The more things we can have that will help patients, the better,” said Messing. “Voice prostheses are the gold standard for patients that have had their larynx removed, but that isn’t every patient. Having more options for patients will only help with their quality of life, and that’s what we all want.”
To make its user’s unvocalized speech audible, the device passes the induced muscle movement signals to a machine learning model running on an external processor. This model is trained to detect patterns of muscle movement that correspond with a fixed number of predefined sentences. When it detects one of these phrases, the processor then plays the sentence back by vibrating the patch’s surface as a speaker. “Similar to how the material converts muscle movements into electricity,” says Chen, “it can also induce electricity as an input signal into mechanical vibrations in the device that produce sound.”
Researchers have developed an AI-powered vocalization patch that is non-invasively adhered to the throat. Thin magnetic induction coils and flexible magnetic materials in it then do the work to infer neck and throat movements beneath the skin. Jun Chen Lab/UCLA
The system’s throat patch applies new material science research from Chen’s group that takes advantage of a property called magnetoelasticity, in which the magnetic field strength of materials changes as the material is stretched and compressed. Regular, everyday activities and neck movements stretch the patch, resulting in magnetic field changes that are then measured by flexible inductive coils built in. These materials work in tandem to passively sense the minute, 3D movements of a user’s neck muscles.
While the magnetoelectric effect has been observed in metallic materials since the 19th century, the stiffness of such materials has made biological applications difficult—including measuring the contraction and expansion of a person’s neck muscles. Chen’s group has found a way to make very stretchy materials magnetoelastic by embedding particles of stiffer magnetoelastic materials into sheets of a flexible polymer.
The new magnetoelastic material’s flexibility lets it adhere to and accurately track the movement of the user’s neck muscles in a way that a similar sensor made from previously known magnetoelastic materials could not. To further enhance its sensitivity, the group shaped the material into a kirigami pattern—a papercraft similar to origami that permits cuts—that made the sensor behave evenly under smaller stretches and deflections.
Moving forward, Chen says the group will work on translating their research into a medical device. “In the future, we want to optimize the device and to standardize the fabrication procedure for mass production,” says Chen. “We have to improve the software and hardware, to increase the translation vocabulary and accuracy and make the device more user-friendly. All of these problems need to be solved, one-by-one.” Chen estimates the research group will produce a viable medical device “in 3-5 years.”
The researchers presented their findings earlier this month in the journal Nature Communications.
surface patch wire wound power inductor Michael Nolan is a writer and reporter covering developments in neuroscience, neurotechnology, biometric systems and data privacy. Before that, he spent nearly a decade wrangling biomedical data for a number of labs in academia and industry. Before that he received a masters degree in electrical engineering from the University of Rochester.