Case Study: The Natural Resources Institute (NRI) used Interactive Voice Response (IVR) Technology in Assessing Dietary Diversity in Northern Uganda

Executive Summary

• Project Goal: The Natural Resources Institute of the University of Greenwich aimed to compare traditional face-to-face diet data collection methods with Interactive Voice Response (IVR) survey calls to assess the reliability and effectiveness of remote data collection in determining dietary quality with hard-to-reach women and children in Sub-Saharan Africa.
• Main Challenges: The study faced challenges including low literacy levels, limited technological exposure, and cultural sensitivities in geographically remote Northern Uganda.
• Participant Engagement: 156 women-child pairs were enrolled in the study. 74% (116 out of 156) of the women successfully completed the IVR survey.
• Call Duration: The IVR survey consisted of a total of 88 questions on dietary diversity and hand-washing practices with skip logic, an average call length of 23 minutes, and an average of 71 questions answered.
• Costs: The approximate cost of the 3-month program is USD$1,491.00 (USD$12.85 per completed survey).

The Natural Resources Institute, University of Greenwich, based in the UK, partnered with the London School of Hygiene and Tropical Medicine (LSHTM) and two Ugandan NGOs: Africa Innovations Institute (experience in nutrition research) and Ichuli Institute (experience in Northern Uganda and with IVR). This project was supported by Innovative Methods and Metrics for Agriculture and Nutrition Actions (IMMANA) programme from LSHTM with funding from UKAid and the Bill and Melinda Gates Foundation.

Background & Goal

Northern Uganda, a geographically remote area, presents unique challenges in public health research, particularly in collecting reliable and timely dietary data. It is particularly challenging to monitor dietary quality during the wet ‘hungry’ season when risk of malnutrition is high for women and children and when some roads become impassable, especially in fishing communities near the River Nile, Lake Kwania, and the surrounding swamplands. Researchers from the Natural Resources Institute, University of Greenwich, in partnership with the London School of Hygiene and Tropical Medicine, (two Ugandan NGOs), Africa Innovations Institute and Ichuli Institute (two Ugandan NGOs), undertook a pioneering study supported by the Innovative Methods and Metrics for Agriculture and Nutrition Actions (IMMANA) programme with funding from UKAid and Bill and Melinda Gates Foundation.

The project titled “Validation of ICTs for understanding women’s time use, dietary diversity, and hygiene practices” was undertaken during the wet season between August and November 2022 in Kwania and Apac Districts of Northern Uganda. Their goal was to validate a novel method of collecting data on women’s and children’s dietary diversity and hand-washing practices remotely using Voice IVR (interactive voice response [1]) surveys via mobile phones in a rural, low-resource, agropastoral setting in Northern Uganda.

The research team chose engageSPARK’s Voice IVR survey platform for its accessibility and ease of use, essential in areas with low literacy rates and limited exposure to technology. The engageSPARK IVR pricing structure also contributed to the cost-effectiveness of the tool.

Implementation and Survey Design

The study involved recruiting pairs of women and children from Northern Uganda. The research team recruited 156 women-child pairs from 13 rural villages across Kwania and Apac districts in the Lango sub-region of Northern Uganda.

• The women were aged between 19-49 years.
• The children were aged between 12-23 months.

The decision to recruit women aged between 19 and 49 years with children aged 12 to 23 months was strategic. The collection of data on this specific demographic is critical for these key reasons:

1. Nutritional Vulnerability: Women of reproductive age and young children are particularly vulnerable to nutritional deficiencies. Understanding their dietary diversity provides insights about the quality of their diets, which is important for the nutritional health of the community.
2. Data Relevance: The data collected from this group is highly relevant for public health interventions. A healthy mother is critical for a healthy baby – in other words, the mother is the main source of nutrition for the fetus in utero and during the exclusive breastfeeding phase when the infant is 0-6 months old. The foods that the young child eats during the complementary feeding phase (6 months to 2 years old) are also critical for growth and development so that the child can do well at school.
3. Long-Term Impact: Focusing on young children’s nutrition has long-term implications for their growth, development, and overall health. Early childhood is a critical period for nutritional interventions to have a significant and lasting impact.

The recruitment process was carefully planned to ensure a representative sample from these remote areas. The team’s approach was tailored to meet the unique challenges of conducting meaningful and high-quality research with hard-to-reach population groups in geographically remote zones, emphasizing the need for reliable data collection methods that can operate effectively under challenging circumstances.

The project used the standardized Ugandan Demographic and Health Survey dietary diversity questionnaires for women and children.  These questionnaires are used in national surveys routinely conducted by national governments with funding and support from USAID and other development partners such as UNICEF.

Village in Kwania District, Northern Uganda where the study was undertaken. All photos in this case study were taken by Lydia O’Meara in 2022, in Northern Uganda.

The team faced several challenges, including dealing with low literacy levels, unfamiliarity with mobile phones, and sensitive issues like gender-based violence around the use of mobile phones. Strategies to overcome this included extensive community engagement, the use of a female voice to reduce the risk of male jealousy and potential harm to women, simplicity in questions and refined translation, literacy and mobile phone usage training, IVR programming for clarity and continuity, timing of the call, and automated call retries to overcome poor network connectivity. Taking time before the survey to find solutions to these challenges with egageSPARK’s assistance was critical. This included solutions such as:

1. Invalid Response Handling: The IVR system was programmed to repeat questions if an incorrect button other than 1 or 3 was pressed (1 indicating ‘yes’, 3 indicating ‘no’).
2. IVR Survey Reconnect: When mobile phone call was dropped during the automated interview because of poor network connection or the participant accidentally pressing the end key button, the engageSPARK system called the women back and reconnected them to the question where they left off.
3. Call Time Windows: The time of day of the calls was programmed using engageSPARK’s call time window feature.
4. Call Retries: engageSPARK’s customizable automated call retries helped overcome poor network coverage.

The average length of the IVR call was 23 minutes. Feedback from participants suggested that this duration was longer than ideal, indicating a preference for shorter surveys of around 10-15 minutes. The survey comprised a total of 88 questions, structured with skip logic to tailor the flow based on responses. Not all questions were asked to every participant; they progressed to different parts of the survey depending on their responses to previous questions. On average, 71 questions were answered per participant.

Results – 74% of the participants successfully completed the IVR survey

The team’s decision to use push button or keypress responses rather than voice responses was based on prior experience in a similar context, where women showed reluctance to speak back to the phone. This approach made the survey process less intimidating and more private for the participants.

Despite facing numerous challenges, the research team achieved significant milestones in terms of participation and data quality.

1. Participant Completion Rate: Close to three quarters of the women successfully completed the IVR survey. This high completion rate demonstrates the feasibility of using IVR technology in remote, hard-to-reach populations.
2. Impact on Dietary Diversity Monitoring: The study’s preliminary findings indicate that information on the dietary diversity of women and children can be collected using Voice IVR surveys during challenging periods like the wet ‘hungry’ season in geographically remote zones. This tool shows promise as a high-frequency, remote monitoring system to detect deteriorations in dietary diversity to identify the need for early intervention to address malnutrition in mothers and children.
3. Cost-effective: Preliminary analysis indicates that the IVR tool is relatively cost-effective as a remote diet data collection tool (compared with costs associated with frequent in-person traditional enumerator methods such as wages, hiring cars, and fuel). For example, the total cost of conducting the IVR component of this study was approximately USD$1,491 (this included $0.13 per minute x 30min per call x 156 women x 2 calls, plus additional call time for sensitization workshop demos, mother’s training calls, and monthly account fees). This translates to USD$9.56 per participant and USD$12.85 per completed survey.

These preliminary results highlight the potential of the IVR technology not only in successfully reaching and engaging a significant portion of a hard-to-reach target demographic but also in collecting meaningful data under challenging circumstances.

Conclusion

The study’s findings indicate that IVR technology can be a valuable tool for high-frequency, long-term monitoring of dietary diversity. It offers a non-intrusive, cost-effective way to collect data remotely, especially in remote or challenging environments like Northern Uganda. This success also depended on the efforts made to teach the women how to use the project phones effectively in a way that made them feel comfortable and confident, and of adapting data collection methods and protocols to safeguard the safety of participants, especially of women at risk of gender-based violence. Many participants were unfamiliar with reading numbers and using phones, necessitating dedicated training sessions. Special attention was given to ensuring participants pressed the buttons firmly enough for the responses to register on the engageSPARK platform.

Example of a basic push button phone used in this study – note the green and the red coloring. This is to help teach the women how to pick up the phone call. The research team taught the participants how to read and press the number 1 for yes and number 3 for no. They found it was important to pick 1 and 3 away from each other to avoid accidentally pressing the wrong button.

Despite the initial challenges, the project was successful in achieving its goal of testing the IVR tool against traditional data collection methods. It shows that this tool could be used as a low-cost alternative for governments, NGOs, and development partners to remotely assess dietary quality among hard-to-reach nutritionally vulnerable groups, such as women and children during the wet (‘hungry’) season when roads are impassable, or other situations when it is difficult to access participants using traditional methods such as during epidemic outbreaks or in conflict zones.

The research team has expressed great satisfaction with the engageSPARK platform and eagerly anticipates future collaborations with the engageSPARK team on upcoming projects. Lydia O’Meara, the study’s postgraduate nutrition researcher, said: “The engageSPARK team helped to capacity build both our skills as ICT researchers and generously worked closely with our local Uganda IT partners to help refine their IVR survey development skills e.g. by providing lots of personalized online working meetings while we developed the survey on the engageSPARK platform.”

The research team is exploring further funding opportunities to use IVR technology in longitudinal studies, underscoring the potential of this approach for affordable remote ongoing public health monitoring and research.

[1] Interactive Voice Response (IVR) is the technology that powers interactive automated phone calls, where humans receive a pre-recorded phone call (or call into a hotline) and interact with the call by pressing keys on their phone’s keypad.  For example, the research team in this study prerecorded the questions it wanted to ask on its IVR survey, uploaded the audio files to the engageSPARK website, and created the survey flow.  The engageSPARK system then called the participants, played back the pre-recorded audio file for the first question and waited for the participant to press keys on their phone to answer the question  (e.g., press 1 for yes or 2 for no); then depending on the participant’s response, the survey might continue to the second question or a later question, or the call could end.  In sum, IVR represents the concept of keypress interactions on a pre-recorded automated phone call.  (Note: IVR also refers to spoken responses, where the participant speaks into the phone and their response is captured and processed).