Decoder uses fMRI brain scans to reconstruct human thoughts

Researchers at the University of Texas at Austin have developed a decoder that uses information from fMRI scans to reconstruct human thoughts. Jerry Tang, Amanda Lebel, Shaily Jain, and Alexander Huth published an article describing their work on the bioRxiv preprint server.

Previous efforts to create technology that can monitor brain waves and decode them to reconstruct a person’s thoughts, all of which consisted of probes placed in the brains of willing patients. And while such technology has proven useful for research work, it is impractical for use in other applications, such as helping people who have lost the ability to speak. In this new effort, the researchers extended the results of previous studies by applying findings about reading and interpreting brain waves to data obtained from fMRI scans.

Recognizing that trying to reconstruct brain waves into individual words using fMRI was impractical, the researchers developed a decoding device that sought to gain a general understanding of what was going on in the head, rather than a verbatim decoding. The decoder they built was a computer algorithm which took fMRI data and returned paragraphs describing general thoughts. To train their algorithm, the researchers asked two men and one woman to lie in an fMRI machine while they listened to podcasts and recordings of people telling stories.

Over time, each volunteer listened to 16 hours of voice recordings covering a wide range of topics. The decoder algorithm analyzed the data from the fMRI machines, processed what it was given, and used what it learned to create predictors that represented textual ideas. The team then used the predictors (guessings) made by the decoder to try to reconstruct what was going on in the subject’s mind based on the new fMRI data sets. To create a measure of accuracy, they developed a scoring system and used it to measure the accuracy of the text generated by the decoder against the fMRI data that was originally generated. The researchers describe their decoder as “pretty good” at matching brain wave activity to the content of the original story.

The researchers also note that their technology could not be used to read human data the mind without their knowledge or consent, because training the decoder must first be done using fMRI data from the people whose thoughts are to be decoded.

© 2022 Science X Network