On May 19, 2021, Dr Walter van Heaven from the University of Nottingham was invited by the lab of our centre to give an online talk entitled “Watching captioned videos: Impact of mismatches between spoken and written information”. The talk was given via Zoom, attracting almost 90 participants.

Dr. van Heuven first pointed out that as an information-rich resource, subtitles are very common in movies and TV shows. A large number of studies in the last 30 years have focused on the impact of captions on foreign language learning and teaching. After introducing a series of empirical studies on caption processing, he proposed the following research questions: To what extent do native and non-native English speakers read English subtitles in an English spoken film? Do viewers notice mismatches/ errors in the subtitles? Does the mismatched information impact memory?

To answer these questions, Dr. van Heuven used the eye-tracking technique to collect data from adult participants under three conditions by manipulating critical words in subtitles: correct words, pseudohomophones, and synonyms. He found that the total time spent on pseudohomophones and synonyms was longer than on correct words, and that the mean fixation duration was longer in pseudohomophone condition. Memory for spoken words was improved when combined with pseudohomophones. In addition, the results of native and non-native speakers were similar. Furthermore, Dr. van Heuven partially replicated his experiment among children. Children spent more time on words with spelling errors than correct words, which was particularly true among children with higher reading skill. More skilled children also spent more time reading subtitles than less skilled children did.

The talk provided a novel perspective and enlightened the audience. During the Q&A session, Dr. van Heuven answered questions at length for the audience.