Scientists Develop A.I. System Focused On Turning Peoples’ Thoughts Into Text

According to a recent peer-reviewed study that was published in Nature Neuroscience, scientists have created a noninvasive A.I. system that is focused on translating the brain activity of an individual into a text stream. This system has been called a semantic decoder- which will be able to benefit patients who have already lost their ability to communicate physically after they have suffered a stroke, paralysis, or some form of degenerative disease. The researchers involved in this project hailed from the University of Texas and developed this system through a transformer model- which is actually quite similar to the one that supports the Bard chatbot of Google, and ChatGPT– the chatbot of OpenAI. [1]

The participants involved in this study trained this decoder by listening to multiple podcasts through an fMRI scanner. For those wondering, this is a large piece of machinery that is used to measure the activity taking place in the brain. This system is absolutely noninvasive and does not need any surgical implants. With the completion of this project, the A.I. system will be trained, wherefore it would be able to generate a text stream when the participant is listening or has thought of something. Although the resultant text will not be a direct transcript, it would still be created with the intent of capturing ideas or thoughts. 

A.I. Decoder Can Accurately Translate Brain Activity Into Legible Text

Another news release posited that this trained system produced texts that were closely related to the intended meaning of the original words of the participants at least 50% of the time. For example, when the participant heard, “I don’t have my driver’s license yet” during the experiment, the thoughts were roughly translated to, “She has not even started to learn to drive yet.” One of the lead authors of this study, Alexander Huth, mentioned, “For a noninvasive method, this is a real leap forward compared to what’s been done before, which is typically single words or short sentences. We’re getting the model to decode continuous language for extended periods of time with complicated ideas.” [2]

Interestingly, the researchers also made sure to address the concerns of how this A.I. technology could be misused. The paper went on to describe that the decoding worked only with the participants who were cooperative and had participated willingly in training the decoder. The results for individuals who had not trained the decoder were completely unintelligible, and if the participants put up even a shred of resistance, the results were deemed unusable. Another lead author of the study, Jerry Tang stated, “We take very seriously the concerns that it could be used for bad purposes and have worked to avoid that. We want to make sure people only use these types of technologies when they want to and that it helps them.”

From Stories To Entire Films – The Decoder Is Pretty Promising

Initially, the participants were only asked to listen or think about stories. But then the researchers made them watch four short, silent videos while they were in the scanner. The A.I. decoder was able to aptly decode the brain activity- which was then used to describe certain events from the videos. Unfortunately, while one can see a major advantage of such technology, it is quite impractical outside a laboratory- since it is entirely reliant on an fMRI machine. But researchers are hopeful that this would be translated into more portable brain-imaging systems. 

Huth remarked, “fNIRS measures where there’s more or less blood flow in the brain at different points in time, which, it turns out, is exactly the same kind of signal that fMRI is measuring. So, our exact kind of approach should translate to fNIRS,” although, he noted, the resolution with fNIRS would be lower.

For more detail on their research check out the the podcast below!

Keep Reading: Scientists Accidentally Discovered New Material That Can ‘Remember’ Like a Brain