Keio University Research: Brainy insights into emotion recognition

TOKYO, Sept. 29, 2021 /PRNewswire/ -- Researchers at Keio University, Japan, describe their research on comparing pre-and post-surgery neural activity for exploring how the brain handles emotions offer and virtual-reality ball-catching experiments that reveal the human brain's capability of simultaneously operating multiple synchronized bodies. These findings have been published in the journals Cortex and Biological Psychology, and the proceedings of the Augmented Humans International Conference 2021 (AHs2021).  

The Keio Research Highlights website offers more details about this and other recent research being conducted by researchers at Keio University.

https://research-highlights.keio.ac.jp/

Emotional awareness: Brainy insights into emotion recognition

Comparing pre-and post-surgery neural activity provides an alternative approach for exploring how the brain handles emotions

"My interest in psychology started during my high school days," says Yuri Terasawa, an associate professor at the Department of Psychology, Keio University. "High school is a time when our bodies go through many physical and emotional changes. I remember thinking why feeling sad caused tightness of the throat and a sense of not being able to breathe easily. And, why do we get the sense of 'moya-moya' in Japanese, where you feel stress but do not know why, but your mind feels foggy and your body feels fatigued? So I decided to earn a degree in psychology to take a deeper look at the interaction of the mind and body that leads to human emotions."

Terasawa is working with surgeons and industrial collaborators on gaining insights into the question of "what is emotion?" using methods that include experimental psychology tests, functional magnetic resonance imaging (fMRI), and wearable devices. "I want to quantify differences in how people feel emotions such as interoception, anxiety, and alexithymia," explains Terasawa. "That is, why do different people experience the same emotions in different ways? This is an exciting area of multi-disciplinary research."

Emotions are processed in a part of the brain called the insular cortex, but understanding how exactly the insular cortex deals with emotions is not straightforward. One difficulty lies in disentangling psychological aspects from 'interoception' — the set of neural mechanisms involved in transmitting signals to the brain. Most research on the relationship between the insular cortex and emotions relies on neuroimaging techniques like magnetic resonance imaging.

However, an alternative approach was reported by Terasawa and colleagues at Nagoya University School of Medicine and Nagoya University Hospital, who describe the effects of surgery on the insular cortex with regard to patients' abilities to recognize different emotions¹.

The scientists analyzed data obtained from 12 patients who underwent resection of a tumor affecting the insular cortex. The patients were asked to perform some tasks before and after the operation. The first task was the heartbeat counting task, a classic exercise used in psychology where a patient is asked to count the number of heartbeats felt in a short period of time, without using the sense of touch. The discrepancy between the number of reported heartbeats (by the patient) and actual heartbeats (measured with an electrocardiogram) can be converted into a measure of interoceptive accuracy.

In another task, named the emotional sensitivity task, patients were shown photos of faces expressing anger, sadness, disgust, happiness or "nothing" (a neutral facial expression). Participants were asked to decide what emotions they felt when shown photographs and a "correct response index" was calculated based on their responses.

From a comparison of the results of the heartbeat counting task before and after the operation, the scientists concluded that on average, the group of patients as a whole did not change its interoceptive performance. However, the results of the emotional sensitivity task showed a decreased sensitivity to anger and happiness after surgery. By applying a detailed statistical analysis of the individual performances on the heartbeat counting and emotional sensitivity tasks, Terasawa and colleagues were able to conclude that the decreased ability to recognize anger and happiness are associated with particular changes in interoceptive accuracy — those patients who had a higher change in error rate for the heartbeat counting task scored worse when having to identify the feelings of anger and happiness.

"Our results confirm firsthand accounts I have heard from close relatives of patients who have undergone resection," says Terasawa. "For example, the wife of a man who had undergone surgery on the insular cortex described the way her husband who had always been excited and happy when he met his daughter after a long while, changed before and after the surgery. He was usually very happy before his surgery, but after surgery exhibited a 'bland attitude,' without visually showing any excitement when he met his daughter."

The researchers point out that there are a few limitations to their study. The number of participants was low, and the influence of the tumors themselves (rather than the surgical operation) remains unclear.  Also, it is not clear whether there are more variables that could have caused the observed relation between interoceptive accuracy and emotional recognition. Nevertheless, the work of Teresawa and colleagues represents, quoting the scientists, "the first study to examine the change of interoception and emotion after insular resection in humans," and it shows that "the removal of the insula[r cortex] affects the recognition of emotions such as anger and happiness through interoceptive processing."

In related research, Terasawa and her colleagues recently described the effects of external stimulation — via haptic signals from customized digital wristwatches — that simulated slowed down heartbeats on the "physiological responses and prosocial behavior" of subjects exhibiting differing interoceptive ability under stress². "The results from our experiments using wearable technology showed that differences in an individual's interoceptive accuracy affected their perception and response to our altered bodily signals. These results show the importance of wearable devices for personalizing biofeedback in this kind of research."

https://research-highlights.keio.ac.jp/uploads/highlights/202109_02_01.jpg

Figure caption: The lesion overlay maps for participants of Terasawa et al. (2021). The insula (colored area) is a key area for considering the relationship between the body and emotions. ©︎ Keio University

References

1. Terasawa, Y., Motomura, K., Natsume, A., Iijima, K., Chalise, L., Sugiura, J., Yamamoto, H., Koyama, K., Wakabayashi, T., & Umeda, S. (2021). Effects of insular resection on interactions between cardiac interoception and emotion recognition. Cortex, 137, 271–281. https://doi.org/10.1016/j.cortex.2021.01.011
2. Xu, M., Tachibana, T., Suzuki, N., Hoshino, E., Terasawa, Y., Miki, N., & Minagawa, Y. (2021). The effect of haptic stimulation simulating heartbeats on the regulation of physiological responses and prosocial behavior under stress: The influence of interoceptive accuracy. Biological Psychology, 164, 108172.
   https://doi.org/10.1016/j.biopsycho.2021.108172

Owning multiple bodies at the same time

Virtual-reality ball-catching experiments reveal the human brain's capability of simultaneously operating multiple synchronized bodies

A human's perception and behavior are coordinated by one and the same body. Yet, the human brain also functions when "connected" to another body as explored in virtual-reality technology settings. This notion is referred to as a "sense of embodiment." An open question is whether a human can have a sense of embodiment for several bodies at the same time: If one human would be able to control multiple (virtual) bodies simultaneously, this could be used for practical applications where they are required to control multiple environments in parallel. The possibility of multiple embodiment has now been investigated by Maki Sugimoto of the Keio University Faculty of Science and Technology and colleagues, who found that up to four virtual bodies can, to some extent, be operated by one person1.

The scientists had 12 people do two types of individual experiments in a virtual environment (VE). The participants wore a head-mounted display on which the VE was projected, and a tracking suit with which the person's motion could be digitally captured and copied onto virtual "cloned" bodies (repeating the posture and motion of the participant) at different locations in the VE. The view of a participant showed the views of the bodies in a matrix-like representation.

In the first experiment, participants had to push a button, turn around and reach for a virtual ball that appeared (which was triggered by pushing the button) from a random direction in the VE. Experiments were done for one, two, and four bodies in the VE. As quantitative assessments of the task, the researchers looked at the distance moved by the participant and the time it took to touch the ball. The former decreased with increasing body number, whereas the latter increased. The researchers argue that this is probably due to the increased cognitive load; with more than one body available, a participant first needs to choose the body to use for reaching for the ball. A qualitative assessment was done by asking the participants a set of questions addressing their sense of embodiment. The main outcome of the questionnaire was that participants applied active-body switching — rather than feeling parallel ownership of multiple bodies, they each time selected a body "to occupy" for carrying out the task.

In the second experiment, four bodies were lined up side by side in the VE, and the participant could see the views of the four bodies (in a 2-by-2 matrix). A virtual ball was then repeatedly thrown to one of the bodies. The target body was selected randomly for each throw, and participants had to catch (by "touching") the approaching ball with the relevant body.

Various interval times were used in the experiment. The success rate decreased with smaller intervals, which again confirmed that participants were switching between active bodies — a conclusion also confirmed by the participants' answers to subjective questions probing their sense of embodiment.

The work of Sugimoto and colleagues is a first step towards realizing the practical potential of distributed embodiment, leading the researchers to conclude that humans can indeed have the sense of body ownership and agency for each body when controlling multiple bodies simultaneously.

https://research-highlights.keio.ac.jp/uploads/highlights/202109_01_01.jpg

Figure caption: The behavior and perception of a participant is synchronized with multiple bodies in the virtual reality environment. © Keio University

Reference and video

 Reiji Miura, Shunichi Kasahara, Michiteru Kitazaki, Adrien Verhulst, Masahiko Inami, and Maki Sugimoto. 2021. MultiSoma: Distributed Embodiment with Synchronized Behavior and Perception. In Augmented Humans Conference 2021 (AHs'21), Association for Computing Machinery, New York, NY, USA, 1–9.

https://doi.org/10.1145/3458709.3458878

About Keio University

Keio University is a private, comprehensive university with six major campuses in the Greater Tokyo area along with a number of affiliated academic institutions. Keio prides itself on educational and research excellence in a wide range of fields and its state-of-the-art university hospital.

Keio was founded in 1858, and it is Japan's first modern institution of higher learning. Over the last century and a half, it has evolved into and continues to maintain its status as a leading university in Japan through its ongoing commitment to producing leaders of the future. Founder Yukichi Fukuzawa, a highly respected educator and one of the most important intellectuals of modern Japan, aspired for Keio to be a pioneer of new discoveries and contribute to society through.

Further information

Office of Research Development and Sponsored Projects
Keio Universit
2-15-45 Mita, Minato-ku, Tokyo 108-8345 Japan
E-mail: keio-rpr@adst.keio.ac.jp

Websites

Keio University
https://www.keio.ac.jp/en/
Keio Research Highlights
https://research-highlights.keio.ac.jp/