Collecting child language data online using eye-tracking
Katie Von Holzen (Technical University of Braunschweig, Germany), Sarah der Nederlanden (University of Amsterdam, The Netherlands) and Sho Tsuji (The University of Tokyo, Japan)
December 20th, 2022
To continue research during the lockdowns of the COVID-19 pandemic, many language development researchers turned to online testing. Online testing can take many forms, but one experimental method in particular - eye-tracking - has recently gained special attention among developmental scientists. These types of developmental studies use eye-gaze to index children’s interest in visual stimuli in response to a language prompt. Eye-tracking has been used in studies ranging from speech sound discrimination over word segmentation and recognition to early sensitivity to syntactic rules. In the laboratory, measuring eye-gaze is either accomplished automatically through a remote eye-tracker or through hand-coding a video recording of children’s eye movements as they view a screen. Compared to such lab studies, online eye-tracking studies offer many potential benefits that go beyond the context of the pandemic, such as accessibility, transparency and efficiency. However, online testing also faces several challenges, including webcam and internet connection quality, potentially noisy, less well controlled testing environments, and safe storage and transmission of video files. Researchers around the globe have come together to work on solutions to these issues, among them the Many Babies - At Home project (MBAH) of which we authors are members (Zaadnoordijk et al., 2021). We aim to support researchers in their adoption of online, eye-tracking methods by establishing best practices in procedure, documentation, standardization, and analysis through global collaboration. Now, 2.5 years after the beginning of the pandemic, we look back at the progress we’ve made and where/how research in this area can/should continue.
In order to present experimental stimuli and collect gaze data from children, several solutions exist. Two such solutions, eBabylab (Lo et al., 2021) and Lookit (Scott & Schulz, 2017), are participant-friendly browser-based platforms that use the caregiver’s webcam to record the child’s eye-gaze. Studies using eBabylab can be hosted on local university servers, which is helpful for data security and protection, but can present a challenge for researchers without such server resources. Lookit is hosted on a central server located in the USA, which makes it easier to access for researchers around the world regardless of their resources, but also requires careful consideration when creating a data management plan.
For researchers considering moving to online data collection, one consideration is whether the measurement is reliable. Participation outside the laboratory might introduce more measurement noise through household distractions as well as issues with internet and webcam quality. Results on these points are mixed. Similar effect sizes have been found for in-lab and online experiments in some, but not all, individual studies (overview in Tsuji et al., 2022). A recent meta-analysis of closely matched replications suggests similar effect sizes across in-lab and online studies (Chuey et al., 2021), suggesting that results from both contexts are comparable. For Lookit specifically, previous studies provide reasons for optimism about the viability of online eye-tracking research (Nelson & Oakes, 2021; Scott et al., 2017). For instance, Scott, Chu, and Schulz (2017) administered online implementations of three previous developmental studies and found that the results of two of three studies were directly comparable to results in the lab.
Online studies have the potential to combat the lack of diversity in language development research (Kidd & Garcia, 2022). As participants do not have to come to the lab, it is possible to recruit people who do not live in close proximity to a lab. When recruitment is no longer local, it also allows for collective solutions. For example, a group of developmental scientists in the USA came together and launched Children Helping Science, a one stop platform for parents and researchers to find online studies to participate in. Kinder Schaffen Wissen (Children Create Knowledge) was launched in Germany to do the same for German-speaking children and parents.
Yet, there is currently little evidence that reaching a more diverse sample can be achieved without active measures. One online, US-based study reports that participants were roughly representative of the American population with regards to income, parental education and race (Scott & Schulz, 2017). In contrast, in another US study participants' racial and socio-economic status remained predominantly white and middle to upper-middle class (Lapidow et al., 2021). Indeed, factors such as access to a stable internet connection and the lack of legal and linguistic infrastructure to expand online studies beyond the Western world still severely limit global access (Tsuji et al., 2022). When recruiting measures are intentionally focused, however, online, remote studies have the potential to reach more diverse samples (Lourenco & Tasimi, 2020), and future initiatives such as offering wifi access and multilingual studies yield the promise of increasing diversity.
By working towards reaching new and more diverse samples, online studies can also help us explore new avenues in language development research. One sub-project of MBAH focuses on the developmental trajectory of word recognition in infants learning different languages, using an online version of the traditional LWL paradigm (e.g., Fernald et al., 2008; Golinkoff et al., 1987) Although this paradigm has been used extensively across languages, comparisons of infants learning different languages within the same study are rare ((Ramon-Casas et al., 2009). An online, remote approach can facilitate cross-linguistic comparisons of word recognition development. However, creating a controlled multilingual and multi-cultural study with comparable linguistic stimuli that is at the same time suitable for young infants proves challenging: how do we select stimuli that are appropriate cross-culturally, and how do we account for any resulting differences between infants learning different languages? This approach has broad implications for the study of language development, as adaptation to the local context remains a challenge for multi-site replication projects, or, indeed, cross-linguistic and cross-cultural work more generally.
In any event, online, remote developmental language studies are here to stay. The pandemic has pushed forward required methodological developments, such that such online studies are now more accessible to the research community. Further developments in areas such as participant recruitment, data management protocols, standardized data collection and automatic gaze coding methods, all of which MBAH works on, are required in order for online data collection methods to fulfill the promise of a more global, more diverse way to conduct developmental studies.
References
Chuey, A., Asaba, M., Bridgers, S., Carrillo, B., Dietz, G., Garcia, T., Leonard, J. A., Liu, S., Merrick, M., Radwan, S., Stegall, J., Velez, N., Woo, B., Wu, Y., Zhou, X. J., Frank, M. C., & Gweon, H. (2021). Moderated Online Data-Collection for Developmental Research: Methods and Replications. Frontiers in Psychology, 12, 734398. https://doi.org/10.3389/fpsyg.2021.734398
Fernald, A., Zangl, R., Portillo, A. L., & Marchman, V. A. (2008). Looking while listening: Using eye movements to monitor spoken language comprehension by infants and young children. In I. A. Sekerina, E. M. Fernández, & H. Clahsen (Eds.), Developmental Psycholonguistics: On-line methods in children’s language processing (Issue 2008, pp. 97–135). John Benjamins Publishing Company.
Golinkoff, R. M., Hirsh-Pasek, K., Cauley, K., & Gordon, L. (1987). The eyes have it: Lexical and syntactic comprehension in a new paradigm. Journal of Child Language, 14(1), 23–45. https://doi.org/10.1017/S030500090001271X
Kidd, E., & Garcia, R. (2022). How diverse is child language acquisition research? First Language, 42(6), 703–735. https://doi.org/10.1177/01427237211066405
Lapidow, E., Tandon, T., Goddu, M., & Walker, C. M. (2021). A Tale of Three Platforms: Investigating Preschoolers’ Second-Order Inferences Using In-Person, Zoom, and Lookit Methodologies. Frontiers in Psychology, 12, 731404. https://doi.org/10.3389/fpsyg.2021.731404
Lo, C. H., Mani, N., Kartushina, N., Mayor, J., & Hermes, J. (2021). e-Babylab: An Open-source Browser-based Tool for Unmoderated Online Developmental Studies. PsyArXiv Preprints. https://psyarxiv.com/u73sy/
Lourenco, S. F., & Tasimi, A. (2020). No Participant Left Behind: Conducting Science During COVID-19. Trends in Cognitive Sciences, 24(8), 583–584. https://doi.org/10.1016/j.tics.2020.05.003
Nelson, C. M., & Oakes, L. M. (2021). “May I Grab Your Attention?”: An Investigation Into Infants’ Visual Preferences for Handled Objects Using Lookit as an Online Platform for Data Collection. Frontiers in Psychology, 12, 733218. https://doi.org/10.3389/fpsyg.2021.733218
Ramon-Casas, M., Swingley, D., Sebastián-Gallés, N., & Bosch, L. (2009). Vowel categorization during word recognition in bilingual toddlers. Cognitive Psychology, 59(1), 96–121. https://doi.org/10.1016/j.cogpsych.2009.02.002
Scott, K., Chu, J., & Schulz, L. (2017). Lookit (Part 2): Assessing the Viability of Online Developmental Research, Results From Three Case Studies. Open Mind, 1(1), 15–29. https://doi.org/10.1162/OPMI_a_00001
Scott, K., & Schulz, L. (2017). Lookit (Part 1): A New Online Platform for Developmental Research. Open Mind, 1(1), 4–14. https://doi.org/10.1162/OPMI_a_00002
Tsuji, S., Amso, D., Cusack, R., Kirkham, N., & Oakes, L. M. (2022). Editorial: Empirical Research at a Distance: New Methods for Developmental Science. Frontiers in Psychology, 13, 938995. https://doi.org/10.3389/fpsyg.2022.938995
Zaadnoordijk, L., Buckler, H., Cusack, R., Tsuji, S., & Bergmann, C. (2021). A Global Perspective on Testing Infants Online: Introducing ManyBabies-AtHome. Frontiers in Psychology, 12, 703234. https://doi.org/10.3389/fpsyg.2021.703234