Case Study: Johns Hopkins University

Johns Hopkins University’s Bloomberg School of Public Health (JHU or JHSPH) is the top international public health school in the world and the largest educational institution of its kind. It is dedicated to improving the health of the global community by deploying diverse medical professionals and scientists across the globe to research and innovate on how to prevent diseases and injuries. JHU then implements largescale solutions to safeguard the public’s health worldwide.

Background

Johns Hopkins University’s Bloomberg School of Public Health (JHU or JHSPH) focuses on protecting the world’s population from diseases and injuries. Its diverse medical professionals, researchers, scientists, and educators “work with communities and populations [to] identify the causes of disease and disability and implement largescale solutions.”

In October 2017, JHU reached out to engageSPARK to explore the possibility of collaborating on public health research in Colombia using SMS text message surveys and Voice IVR (interactive voice response[1]) surveys.  engageSPARK is a social enterprise that offers a browser-based, cloud-hosted platform used by non-techies at NGOs and businesses to launch interactive Voice IVR and SMS text message campaigns globally.

The project, The Bloomberg Data for Health Initiative, is funded by Bloomberg Philanthropies and the Government of Australia. It is divided into two components:

• improving civil registration and vital statistics.
• improving data use.

To achieve some of the project’s goals, JHU employed the use of mobile phone surveys for real-time data gathering. The surveys are divided into three experiments:

• Experiment 1 is to identify the appropriate incentive amount to motivate participants to complete an IVR survey.
• Experiment 2 is to identify the appropriate incentive amount to motivate participants to complete an SMS text message survey.
• Experiment 3 is to determine appropriate methods for participants to consent to IVR surveys:

The incentives were in the form of airtime top-ups (prepaid mobile phone credit).  Each IVR survey and SMS survey began by offering participants a specific amount of airtime to complete the survey, with different participants offered different incentive amounts.

This case study focuses mainly on Experiment 1 of JHU’s public health survey for non-communicable disease risk factors.

Experiment 1 of the research has two primary goals:

• to determine what amount and type of incentive would encourage people to take and complete an automated phone call (IVR) survey.
• to find out whether an IVR survey is an effective method of collecting health information from people in different countries.

JHU sought to survey people from different walks of life in Colombia and wanted the sample to be truly random, so JHU used a random digit dialing (RDD) approach to generate millions of potential Colombian phone numbers, with the aim of collecting 10,000 to 15,000 complete surveys over the course of 14 months.

JHU rolled out the first set of mobile phone surveys in September 2018. The IVR survey consisted of approximately 25 questions that took 8-10 minutes to complete. The beginning of each call informs participants what incentive they will receive if they complete the survey.  Participants are broken into three incentive groups: (a) some participants are offered 5,000 Colombian Pesos (COP) (approx. $1.55 USD) of prepaid mobile phone (airtime) credit; (b) others are offered 10,000 COP ($3.10); and (c) the third group is told that they will be enrolled in a lottery where one in twenty people who complete the survey will receive 50,000 COP ($15.50).  A fourth group, the control group, receives no incentive.

On September 29, 2018, the JHU team, sitting in their offices in Baltimore, Maryland launched the first set of IVR surveys from the engageSPARK website to 15,000 people in Colombia: 5,000 for each of the three incentive groups. The engageSPARK system called all 15,000 people, played back the survey questions, and collected the keypress responses.  The JHU research team was able to see a summary report on the engageSPARK website showing the aggregated survey responses (e.g., 50 people pressed 1 on question 1, 100 people pressed 2, etc.), and download a detailed Excel report showing how each person answered all of the questions in the survey.

JHU continued to launch tens of thousands of IVR surveys every day until Experiment 1 ended in mid-November 2018. Participants expressly consented to JHU using the data collected in the survey, and they answered questions about their age, health, health behaviors, vices, educational background, diet, etc. by pressing keys on their mobile phone keypads (e.g., 1 for yes, 2 for no). At the beginning of the call, participants were asked for their age because only those who are 18 or older were permitted to proceed in answering the rest of the questions.

Results

The data JHU collected via the engageSPARK-powered IVR surveys in Colombia was comparable to results JHU received from running the same survey in other countries. However, we will not be able to disclose any additional information about the surveys until JHU completes its analysis and publishes its own report.   JHU is hoping to be able to answer questions like:

• Will people take a survey asking questions about their health and health behaviors?
• How expensive is it to conduct these surveys?
• Who are the people who take these surveys?
• Do participants look like the general population of the country? If not, are they younger, older, more educated, less educated, etc.?
• Do the results from these surveys look like the results they get from door-to-door surveys that are conducted? If not, why not?

We look forward to publishing a more detailed case study – discussing answers to the above questions and outlining the actual survey results (number of total calls, completed surveys, how long it took to complete the surveys, costs per completed survey, etc.) – once JHU concludes its analysis and publishes its report.  Those results will also help research practitioners estimate what kind of results they can expect from running random digit dialing (RDD) surveys.

Conclusion

By collaborating with engageSPARK, JHU was able to gather random data sampling of the general public’s age, health, health behaviors, vices, educational background, diet, etc. in Colombia through Voice IVR survey. The success of the project’s initial mobile phone survey experiment paved the way to the continuation and completion of the second experiment (SMS survey) of “The Bloomberg Data for Health Initiative” project.  JHU is set to collaborate with engageSPARK on yet another project in the fourth quarter of 2019 once its grant is approved and finalized.

[1] Interactive Voice Response (IVR) is the technology that powers interactive automated phone calls, where humans receive a pre-recorded phone call (or call into a hotline) and interact with the call by pressing keys on their phone’s keypad.  For example, JHU prerecorded the questions it wanted to ask on its IVR survey, uploaded the audio files to the engageSPARK website, and created the survey flow.  The engageSPARK system then called the participants, played back the pre-recorded audio file for the first question and waited for the participant to press keys on their phone to answer the question  (e.g., press 1 for yes or 2 for no); then depending on the participant’s response, the survey might continue to the second question or a later question, or the call could end.  In sum, IVR represents the concept of keypress interactions on a pre-recorded automated phone call.  (Note: IVR also refers to spoken responses, where the participant speaks into the phone and their response is captured and processed).