The James S. McDonnell Foundation, known internationally for supporting research on cognition and behavior, has announced that John Franchak, an associate professor of psychology at UCR, is a recipient of a 2022 Opportunity Award Grant.

The three-year, $250,000 grant will support a project focused on understanding infants’ everyday experiences. For the project, Franchak developed a new technique involving the participation of parents of infants. Parents receive 10 randomly distributed prompts every day to take a 10-second video clip of their baby.

“We will be developing a machine learning algorithm to automatically detect things about the baby, such as whether they are sitting versus standing, and will be scoring other things about them, like which objects they are playing with, to see how those experiences predict language learning,” Franchak said.

Franchak explained that everyday experiences in the home that promote infants’ learning — for example, how encountering objects like cups and balls helps them learn the words “cup” and “ball” — are generally difficult to study. 

“Having a researcher enter the home and observe infants for an hour can give only a brief report of what’s happening at a single time,” he said. “It’s not enough, however, for understanding how experiences over days and weeks are accumulated to help infants learn.”

By using a video ecological momentary assessment, or vEMA, Franchak will be able to sample infants’ behavior every hour across two weeks by asking caregivers to take short videos of their baby. This new technique will allow his lab to determine how different kinds of behavior vary over time and help answer questions such as: Do babies have days where they walk a lot or a little? How frequently do babies encounter the same object (like a cup) daily that would help them learn the word “cup”?

“We don’t know much about how frequently infants perform a new skill after learning it,” Franchak said. “A naive way of thinking about learning a new skill like walking is that once a baby can walk, they will walk every day. But because of the limitations in recording infants’ experiences, we haven’t been able to measure day-to-day variability in skills like walking. A newly walking baby might walk one day and then not the other, or they might walk each day after learning to walk.” 

Franchak’s lab has already been using EMA with text messages asking parents to fill out surveys multiple times a day about what their infants are doing. Although that helps his lab understand patterns about everyday experiences, having videos will let his team extract more information about infants’ daily lives, such as what position they are in, what object they are holding, and what room they are in. 

“One issue with collecting video data is that someone usually needs to annotate it — watching frame by frame to manually score what position infants are in or what object they are holding,” Franchak said. “Recent developments in computer vision and machine learning are starting to make it possible, however, to do some of these things automatically. For example, pose detection algorithms can look at a video of a baby and find the locations of every joint automatically with high accuracy, which gives us an automatic way of scoring infants’ motor behavior from video.”

Franchak expects one graduate student will work on the research project.

The James S. McDonnell Foundation is a St. Louis, Missouri based private philanthropic institution established in 1950 by aerospace pioneer James S. McDonnell to “improve the quality of life.” It supports scientific, educational, and charitable causes locally, nationally, and internationally. Since its inception, the foundation has awarded over $450M in grants.