Assessing Early Childhood Responsiveness in Pakistan: A Student Grant Report from Elizabeth Hentschel ’26

From left to right, Saima Siyal (Co-Principal Investigator and Local Field Manager), Saima Solangi, Shehnaz, Elizabeth, Shamim, and Zarina, (Saima Solangi, Shehnaz, Shamim, and Zarina were the research assistants with the DREAM team).

Elizabeth Hentschel, a doctoral student in the Department of Global Health and Population at the Harvard T.H. Chan School of Public Health, spent this past summer working in-country in Pakistan on a dissertation research project, “Measuring Nurturing Care: A Pathway to Healthy Child Development and Protection.” Funded in part by a Mittal Institute Summer Research Grant, Elizabeth’s project took her to Naushahro Feroze, in the Sindh Province of Pakistan, where she spent a month at a child development research site. She led the efforts to create two evidence-informed measures for assessing responsive care and early learning for children. She shares her experiences in the following account. 

Apply for a Student Summer 2022 

Thanks to the generous support of a Lakshmi Mittal and Family South Asia Institute grant, this past summer I collaborated with DREAM NGO in the Naushahro Feroze District of Sindh, Pakistan to conduct an extensive mixed-methods validation study at an established child development research site. In collaboration with the World Health Organization (WHO), as part of my dissertation research, I led the efforts to create two evidence-informed measures for assessing responsive care and early learning.

For each tool, we reviewed the literature, systematically mapped out any existing tools by their broad constructs, iteratively identified common constructs, short-listed to ensure measurability, and piloted on existing data. The final indicator tools are informed by conceptual and operational definitions that describe responsive care and early learning as two distinct concepts that can be interrelated. Prior to these tools, no existing measure adequately distinguished between these two concepts, leaving programs and governments unable to adequately monitor them as separate and distinct entities.

The next step is to thoroughly validate the tools in order to ensure reliability and validity, to update the tool, and ultimately to disseminate the tools worldwide. The funding from this summer grant allowed me to validate these tools, quantitatively and qualitatively. As such, I spent four weeks in Naushahro Feroze this past summer, collaborating with a local co-principal investigator and leading a research team through the training, field testing, and piloting of these tools. 

Week 1 Highlights

Last summer when Dr. Yousafzai, Dr. Aboud, and I developed an indicator for measuring responsive care, one of our primary goals was to create a universal tool that did not positively bias interactions from western contexts. The work of Mesman et al., 2018 draws attention to the fact that the bulk of research on sensitive responsiveness has been carried out on parents in western countries in urban areas, with very little work done in non-western rural regions. They argue, based on their work in Mali, the Republic of Congo, and the Philippines, that sensitive responsiveness is observed in non-western contexts, but it often looks different. Therefore, if we fail to acknowledge that the western conceptualization of responsiveness (which often has an extraverted focus, with more positive affect and vocal exchanges) does not apply to all instances of responsive behavior, then we likely will discredit a range of subtle, and more physical responsive interactions that are more common in non-western settings. As such, in the validation process of our responsive care tool, a strong qualitative component was necessary. Our tool has the three main domains (responsive interactions, non-responsive interactions, and negative interactions), which we expect to be consistent across contexts, but the examples we provide on the tool will be very context specific.

On Sunday, Saima, the co-principal investigator and local field manager, kicked off the qualitative work and held individual in-depth interviews with a subset of purposively sampled women in order to understand how responsiveness is conceptualized in this context. After transcribing and translating the interviews, Saima and I discussed the findings and updated the examples on our tool accordingly. On Monday, Saima and I co-led the first training on how to use the tool. The training was very interactive and hands-on, and was composed of about 25% didactic instruction and 75% learning through doing. Our Research Assistants (RAs), who are very well versed in Early Childhood Education (EDC) work, were quick learners of the new tool. Tomorrow we are excited to start our first day of field testing to see how this tool works in the real world and to figure out what needs to be adapted before we begin piloting! We are excited to continue this work!

After finishing training on Monday, our research team spent Tuesday through Thursday field testing the responsive care tool and spent Friday through Saturday piloting it. These two steps are very important in any research project, as they allow us to work out all issues and quirks the tool has before we initiate the formal data-collection process. Since our tool is observing caregiver-chid interactions during play, we provide each caregiver and child with a toy to facilitate that interaction. The toy has to be age-appropriate, stimulating enough to elicit a reaction from the child, and something they haven’t seen before so that it doesn’t bore them. During field testing we went through a trial-and-error process of figuring out which toys to use for each age group. We also spent a lot of time observing and live coding the caregiver-child interactions while video recording them, and then going frame by frame to discuss any discrepancies in our coding. 

By day three, all of our coders scores were within 10% of each other, and we moved to the pilot phase. We are piloting this tool on 20 caregiver-child pairs, with four of the interactions coded by both a RA and Saima, to check for inter-rater reliability (IRR). IRR refers to the degree of agreement between our coders. We need this to be high, because our RAs will be visiting households independently, and we need their coding to be consistent in order to get reliable results. Based on our first day of piloting the IRR between each RA and the gold standard was between 0.88-0.90, which is considered to be an excellent IRR.

Saima and I have been ecstatic that the training, field testing, and pilot study have been successful so far. Next week, we will begin the training and field testing for the other tools we will use in our study. I am so thankful for this highly motivated team who has worked through this extreme heat! We have been in the field in 90-120 degree weather all week, but no complaints here as the scenery has been amazing and our study participants have been so gracious to welcome us into their homes.

Shamim, Shehnaz, and Zarina, three of our research assistants, posing for a picture outside a village visited for field testing.

One of the many mother-child interactions that we observed while testing our tool to measure responsive care.

Week 2 Highlights

After a successful first week, I’m super excited for week number two. This week will predominantly be spent on training of the child assessment tool. In global health, there are two main types of tools that we use to measure child development: individual measures and population measures. Population measures are useful for understanding the overall child development status of a community, country, or region. These are often inexpensive, quick to administer, and cross-culturally valid. The major drawback of population-level assessments is that they are often imprecise, and may not catch the most nuanced differences in development among children. Therefore, for our study, we are using an individual-level assessment known as the Bayley Scales of Infant and Toddler Development, fourth edition (BSID-IV). This tool is very precise, and the third edition has been used in this setting before. However, it is quite expensive and resource intensive. It takes between 15 minutes to an hour to administer, and requires bringing a subset of toys in a small briefcase to each child visit. As such, we are dedicating this week to training on the tool, which includes four days of classroom trainings and two days of training and testing in the field. 

I also want to express how lucky I feel to be able to collaborate with DREAM NGO, which stands for the Development and Research for Children in Early and Adolescent Years of Life. Saima, my co-principal investigator, has both a master’s degree in education and over 15 years of experience conducting child development research and implementation work. After working as a field manager in Naushahro Feroze for Aga Khan University for 13 years, she has decided to start her own non-governmental organization dedicated to child development work. As an external researcher, working with DREAM has been such an incredible experience for a variety of reasons.

First, our research team is entirely made up of individuals who live and work in Naushahro Feroze. As such, they have developed a great deal of trust with our study participants and community leaders. Second, in addition to being able to employ a very strong team of RAs, all of our research activities directly contribute to the local economy. As this validation study requires a relatively large number of study materials, we are able to employ local carpenters and toy makers to produce them, local restaurants for our study team meetings, and even local grocery stores and markets when buying incentives such as juice and candy for our study participants. In a lot of research projects, there are strict rules about who is allowed to provide the study materials, and they often are imported from abroad or from another region, so we feel very lucky that our work is contributing to this community in every way.

Lastly, DREAM will continue to host and partner with future research groups, so this work is sustainable. All of the training that our RA’s are receiving for this project will be important for future work. For example, they are one of the first groups of RA’s (if not the first) to receive training in the fourth edition of the BSID. After this experience, I definitely will recommend partnering with DREAM to my colleagues interested in child development work.

The full research team training on the Bayley’s Scale for Infant Development in a local restaurant.

Shamim observing a mother-child interaction during the field testing of the responsive care tool, the grandmother is also present.

After two productive days of BSID-IV training on Monday and Tuesday, everyone had the day off on Wednesday and Thursday to celebrate the Eid holiday! This Eid holiday, Eid al-Adha, is the second Eid holiday of the lunar year. This holiday is known as the “feast of the sacrifice” and comes at the end of the Hajj, which is an annual pilgrimage taken by Muslims at the holy city of Mecca. During this celebration, goats, cows, and buffalo are sacrificed in the name of Allah.

I felt very lucky to be invited to spend the holiday with Saima and her family in her village. Saima explained to me that in their village, when someone buys an animal to sacrifice, they keep one-third of the meat, share one-third of the meat with their family on the day of Eid, and give one-third of the meat to the poor. The notion of giving to those less fortunate is a central theme of the Eid al-Adha holiday. During this celebration, it’s also common for women to dress up and to get henna tattoos drawn on their hands, which was a festivity I was excited to take part in! We ended the day with a dip in the irrigation pump down the road, which Saima fondly refers to as a “Pakistani swimming pool.”

Once the Eid celebration was over, we went straight back to work on Friday and Saturday to field test the BSID-IV tool. As the tool is rather time-intensive, we spent a lot of time working out the quirks and figuring out how to present everything in a way that was interesting to the child and not too tedious or tiring. After one more day of field testing on Monday, we will run the full operational pilot with all of our tools on Tuesday and Wednesday. This process has been such an incredible learning experience for me and the team, and I’m excited to see it all come together this week before I head back to America!

One of Saima’s nieces all dolled up for the Eid holiday.

Week 3-4 Highlights

I am so proud of how far our research team has come but so sad to be leaving Pakistan after an incredible month. Although our field team is familiar with the BSID third edition, the fourth edition has a lot of new changes that have been a bit challenging for the team to adapt to. Since our team primarily speaks Sindhi and Urdu and the tool is in English, we have had to translate the items so that they are more accessible for our team members to use. Translation of study tools is tricky, since there are a lot of nuances that cannot easily be translated verbatim. Despite these challenges, our team is quickly becoming experts on the tool, and after collecting some inter-rater reliability data we are ready to begin the formal data collection process!

On my last day with the team, I was showered with gifts, from a blanket that Saima’s mother hand knit for me to a traditional bread basket from Naushahro Feroze. I can’t emphasize enough how incredible this field team is. Employing researchers that live and work in the district has opened doors that would never have been possible had we had a different team. There are so many cultural nuances and lessons that I have learned through them and continue to learn from them. Saima and I both see this partnership as something that will continue throughout our careers, as Saima continues to grow DREAM and I become a more senior researcher. During these last couple of days, Saima and I scheduled a time to meet as a team each week to discuss any issues that occur during data collection and/or reflections from the field. I plan on writing two dissertation papers from this data, and I am excited to involve the team in every step of the process.

Two of the Research Assistant’s children took the group around their village to various households for piloting and field testing of the tools.

More of Elizabeth’s Photos from Pakistan