Hidden joy, sorrow halved
Psychologists from Adlershof conduct basic research on emotion perception and wearing face masks
A wrinkled brow, a frowning mouth, a deepening of the tiny wrinkles around the eyes: More than 40 muscles are involved in forming our facial expressions. Emotion researchers summarise the muscles that are responsible for certain parts of a facial expression into so-called action units. “We have to look at the configurations of action units to classify a facial expression using facial recognition software—which parts of the face move at the same time and how,” explains Till Kastendieck, the group’s research associate. When he and his colleagues from the Organizational and Social Psychology research group at Humboldt-Universität zu Berlin analyse facial expressions, they put the action units in relation to each other and calculate a type of score. “That way we can say things like: There was a tendency towards more positive than negative activity in the face. However, the facial expression and the underlying emotion don’t always have to match each other exactly.”
In non-pandemic times, the team of psychologists also investigated facial movements that are undetectable using sight alone: These are most subtle muscle activities that can be captured only using sensors attached to someone’s head. Just like visible facial movements, these invisible reactions are part of a phenomenon that is frequently studied by the Adlershof-based researchers: so-called facial mimicry. “When people look at another person’s face, they spontaneously and, to a great extent, subconsciously imitate its emotional expression—this is a process that takes place on the borderline between the visible and the invisible,” explains Kastendieck. “Together with Ursula Hess, our department chair, and my colleague Stephan Zillmer, we set out to investigate whether the emotion of our counterparts in the experiment is also recognised and imitated, i.e., whether visible facial mimicry continues to work, if parts of the face we are looking at is covered. Moreover, we wanted to find out whether wearing a face mask changes the intensity with which our subjects experience the emotional expression of another person.”
A woman with a neutral expression begins to smile slowly and cautiously and, in the end, is beaming all over her face. A supermarket can be seen in the background. In another scene, set in a park, a man looks increasingly sad. “The test subjects watched these scenes on a running webcam, we recorded this, and software programme automatically analysed their facial activity,” says Kastendieck. “Immediately after watching the video clips, the test subjects stated, among other things, which emotion they recognised from the person in the video and how close they felt to this person.” The results: People saw less movement associated with joy in the faces of people wearing a face mask than those who saw unmasked people. Further, the subjects also experienced emotions from masked persons, whether happy or sad, with less intensity, and felt less close to the people they were shown. “In the case of grief, however, we found no difference regarding facial mimicry between sad facial expressions of masked or unmasked persons.”
What does this mean for our everyday life? Do the masks limit our ability to share joy and feel close to each other? “In the experiments, the differences are clear. It is important to stress, however, that these basic research results not necessarily have much to do with our everyday life,” says Till Kastendieck. “In everyday life, the informational richness is much higher. Information is conveyed through several channels: through speech, for example, eye expressions, posture, and gestures.” When information is missing on one channel, we can typically compensate for them—by listening to the tone of voice, for example, on the phone. According to the psychologist, the study could be an incentive to think about the richness of communication and about the many possibilities to better respond to a person in a “masked” conversation. “If I wish to avoid expressing anger, for example, I could pay more attention to how I use my voice and what I express with my posture.”
By Nora Lessing for Adlershof Journal