Finding New Ways to Increase Response Rates of Census Bureau Surveys

The U.S. Census Bureau collects data about people and the economy. While most of us are aware of the decennial census that has been conducted every 10 years since 1790, the Census Bureau also conducts over 130 smaller-scale surveys of households and businesses, which are critical in providing important economic and social data that help in policymaking. 

However, the number of people who fail to participate in household surveys has doubled over the last decade. Now more than ever, the Census Bureau is looking for new ways to address respondent concerns and increase participation rates in household surveys.

This is no easy task, but our team of Harvard students was excited to jump in and help them find a solution. Over the last four months, we conducted over 30 interviews, ideated 100 prototype ideas, and tested 13 iterations of those prototypes, arriving at five final recommendations for the Census Bureau to address respondent concerns. Even though we didn’t know much about the Census Bureau at first, our “lack” of experience turned out to be our biggest advantage, enabling us to come up with out-of-the-box solutions that the Census Bureau might never have considered.

A Survey Respondent’s Journey

Typically, we think of completing a survey as a single point in time when a person responds to a set of questions. However, as we spoke with the Census Bureau and potential respondents, we realized that there are various points of interaction that a potential respondent has with the Census Bureau, beyond the act of filling out the survey. We divided these interactions into three stages: pre-survey, a person’s prior knowledge about the Census Bureau and the initial notification that their household has been selected for a survey; survey, the action of completing (or being pursued to complete) the survey itself; and post-survey, a person’s reflection on the survey and, by extension, the Census Bureau overall.

An overview of a survey respondent’s journey through a Census Bureau survey

Breaking down a respondent’s journey through the survey process into these specific stages and interactions allowed us to look carefully at how and where the Census Bureau is currently engaging with respondents. Further, as we ideated and tested new solutions, we made sure to map our prototypes across the entire respondent journey to see where we could create the most impact.

Our Recommendations

The Census Bureau currently has many effective solutions to help people who have been selected for a survey during the main survey stage. However, we recommend that they think bigger, connecting more with respondents during the pre- and post-survey stages to ensure that respondents feel supported throughout the entire process. In addition, as any American household could be selected for a survey someday, the Census Bureau should engage with the US population as a whole to build general goodwill and awareness about household surveys and the Census Bureau’s mission.

Our proposed strategy for the Census Bureau, with a wider approach on who to target and when to act

Based on our user testing with our prototype and conversations with the Census Bureau, we recommend the following action steps to expand their engagement:

1. Implement behavioral science nudges in communication with respondents.
   When a household is selected for a survey, the Census Bureau sends a letter to let the residents know. To combat concerns about legitimacy and increase the likelihood that people see and remember the content of the letter, we recommend adding and highlighting motivating language that has been found effective by behavioral science research.

2. Redesign the website with respondents in mind.
   The Census Bureau’s website has a lot of useful information for survey respondents, but it can be difficult to find answers to specific questions. A simple layout with less text that is readable on both desktop and mobile screens will help prevent information overload.

3. Follow up after the survey.
   After completing a survey, people may have no idea what happens to their response or if anything even comes out of their participation. By following up with respondents after a survey about how their response makes an impact, the Census Bureau can generate goodwill towards the agency overall.

4. Market the existing education curriculum.
   Lack of awareness about the Census Bureau and its surveys can cause people to hesitate to respond to a survey request. One important way to increase this awareness is to start early with primary and secondary school education. The Census Bureau already has a great curriculum for teachers to use, but it is not very well-known, so we recommend that the Census Bureau reach out to schools and other organizations to increase awareness and adoption of these lesson plans.

5. Leverage the decennial census.
   The decennial census has a wide reach and recognition among the people. The Census Bureau should take advantage of the advertising or distribution of the decennial census to increase awareness of other household surveys.

Next Steps

Our group presented detailed findings for the U.S. Census Bureau in Washington DC in May. Our presentation can be viewed here. The team at the Census Bureau is already making strides in implementing some of our recommendations. We are excited that our efforts might influence respondents to participate in household surveys and help the American community.

Alison Chen, Jessica Nunez, Mahesh CR, Tiffany Yu, Vishnu Rajeev

Nudging People to Respond to Census Surveys

In less than a year, the U.S. Census Bureau will conduct the largest peacetime mobilization in the United States to conduct the 2020 Census. Old or young, rural or urban, no matter who you are, you will be counted. But every day, the Census Bureau field representatives are knocking on doors to conduct over a hundred smaller-scale surveys. These household surveys collect important data to inform policies and decisions that affect our communities.

Yet too often, people refuse to respond. 

Our team of Harvard students is working with the Census Bureau to increase response rates so the Census Bureau can collect complete and accurate information. We conducted 27 interviews with Boston-area residents and discovered five major reasons why people don’t respond to Census surveys: lack of awareness about household surveys, concern over the legitimacy of the survey and Census Bureau representative, distrust in government, fears about privacy or data breaches, and people simply not being in the mood to respond. 

Brainstorming Over 100 Ideas

We brainstormed with an open mind and came up with some wild ideas — from partnering with celebrities to endorse surveys to getting rid of household surveys altogether. We thought of ways to augment what the Census Bureau is already doing, as well as completely new ideas they had never considered before. In the end, we came up with over 100 ideas!

We grouped our ideas into similar themes and showed them to our Census Bureau partners, Lisa Clement and Ruth Chan, who helped us narrow them down to the five most interesting, implementable, and impactful solutions.

When we suggested creating new educational curricula involving the Census Bureau to teach students in school, we learned that the Census Bureau already has these resources. Yet they are not well-publicized and the public-school teachers we spoke to had never heard of them, though they would be excited to take advantage of existing lesson plans.

We ended up with four major ideas:

• Redesigning the letter the Census Bureau sends to notify survey respondents

• Improving respondent support pages on Census.gov

• Following up with a thank you message for survey respondents

• Notifying 2020 Census respondents about the existence of other surveys

After creating simple prototypes, we took them out into the hands of potential respondents – everyone around you! — to see whether they would make a difference. 

The Letter

If you are selected for a household survey, you will first receive a letter addressed to your home, explaining which survey you are selected for and notifying you that a field representative will visit sometime soon. In our research, we realized that many people found the letter confusing, difficult to read, or just threw it out before opening it. We wanted to design a new letter incorporating many ideas, from adding more colors and graphics to testing behavioral science theories—psychological, cognitive, and social factors in how humans make decisions.

We tested four different behavior change theories:

Vividness: Humans tend to skim rather than read—and a letter from the U.S. Census Bureau would seem to be particularly susceptible to people skimming it. Therefore, we highlighted in bold text the most critical information in the letter.

Loss framing: Humans are motivated by avoiding economic and other losses. Loss framing messages, in this context, would attempt to motivate respondents who are unaware of the societal issues associated with not completing the survey.

Social norms: Humans are motivated by peer behavior and pressure. When we know everyone else is participating in an activity, we’re influenced by that fact.  We tested messages that informed the respondents that X% of the neighbors had answered the survey. This puts additional pressure on the respondents as would not want to be seen as the very few who do not answer.

Plan making: Behavioral science shows that humans are more likely to follow through on a course of action if they have explicitly made a plan. We wanted to test messages that encouraged people to make a plan to respond to the survey.

Additionally, we included a QR code (a machine-readable matrix barcode that smartphones can decipher, and load a website) to prompt potential respondents to visit the website.

We asked interviewees whether they preferred the original letter or the new one and why. We spoke with Kyla Fullenwider, the former Chief Innovation Officer at the Census Bureau, who has conducted extensive research on survey materials. Kyla’s advice and our own research will inform our next prototype, as we continue to iterate and test our new letter design.

One iteration of the letter prototype

Respondent Support Content on Census.gov

The Census Bureau’s website has several pages of support resources available to help respondents along the survey process, but this information may not be clearly communicated to people browsing through. We rearranged and reformatted the content on one of these support pages to test what details stand out most to a respondent looking for help. In our prototype, the first major section helps a person figure out whether or not they have been selected for a legitimate survey. Meanwhile, the current website emphasizes pictures and links to answers to common questions about legitimacy and data security, as well as Respondent Advocates, who are the primary Census Bureau members responsible for addressing respondent concerns. The Respondent Advocates are currently revamping this page as well, and moving forward, we will incorporate aspects of their drafts into our designs to test.

The top of the current webpage for respondent support and the top of our prototype for the same page.

From our preliminary tests, the Census Bureau got it right. Potential respondents appreciate large, direct links that address their most important questions right away. However, it is not clear to them who “Your Advocate” is, but after learning more about the Respondent Advocate, they appreciated the existence of the service and felt that it would have been useful to know that information right away, rather than having to go to a separate page to find out more.

Following up after a survey

After completing a survey, some respondents report feeling as though their information disappears, with no idea about what happens with their responses. While some field representatives send thank you cards, there is no established communication to acknowledge the important service that respondents have completed and thank them for their time. To address this, we created prototypes of a text message and a thank you card to see whether they might encourage respondents to feel positively towards the Census Bureau, increasing their likelihood to answer another survey or tell their friends about household surveys.

Our text message is short and simple, including a link to a website with more information on the survey they just completed and a fun infographic about the results of the survey. This webpage emphasized that the impact that the survey had could not have been possible without the respondent’s contribution, hoping to show that their response made a direct impact on their own community. Since our key information was on this separate webpage, we tested whether respondents would click on the text message link to learn more.

For those who did not own a cellphone or might prefer a more analog, physical memento, we created a thank you card prototype. The card includes the same infographic as the website, again creating that direct connection to emphasize the importance of answering household surveys. We also tested whether a QR code would be helpful for respondents to get more information online.

Prototypes of the follow-up text message and the physical thank you card

Our initial tests revealed that the colorful infographic of the thank you card was more exciting and eye-catching than the sparse text messages and the lengthy explanation on the website of how the government uses survey data. The potential respondents we tested with so far both described themselves as “analog people” and preferred the physical thank you card over the text message, adding that they would only go to the website if they saw the text message at a convenient time. We’re interested to see how a greater diversity of potential respondents feel about the different message length between the two prototypes and their preferred interactions between the communication channels.

Outreach through the 2020 Census

 Many people already know about the ten-year census, particularly the upcoming 2020 Census. Since everyone is a participant, there is more advertising and general awareness about the importance of this survey. Taking advantage of this knowledge, we created a short acknowledgment at the end of the survey to explain that the Census Bureau’s work does not stop there. In behavioral science classes here at Harvard, we’ve learned that actively acknowledging a task makes individuals more likely to remember to do it later. Since the new Decennial Census will include an option to respond online, we created the following prototype:

Including a message at the end of the 2020 Census can improve awareness of other surveys

Our testing indicates that this message is informative but has an unclear effect on likelihood to complete to surveys. We learned that the prior question influences response and people are concerned about potential solicitations they may receive from acknowledging this message. We iterated on the prototype based on these findings. We are excited to speak to more people to see whether these patterns hold.

Next Steps

We plan to keep iterating and testing to create practical recommendations for the Census Bureau. Along the way, we will talk to more Census Bureau experts to understand how our solutions can be implemented. The Census Bureau is already exploring some of our ideas, including a website redesign and follow-up text message. We are excited to continue working together to make our dreams into reality.

Have you completed a household survey? Then we want to speak with you! Please fill out this 1min form or email vir837@student.hks.harvard.edu.

A Lesson in Rejection: Researching the Household Survey Experience

Augusto is 58 years old. He moved to the United States in 2008, and he isn’t very comfortable with English. When asked whether or not he would respond to a mailed request to complete a survey conducted by the U.S. Census Bureau, he said, “I have to be careful. I need to understand the question and what it contributes to and what improvement is needed.”

There are many people like Augusto in the United States who have reservations about answering these demographic surveys. However, the Census Bureau needs their information to provide up-to-date statistics about the American population to allow organizations to make informed community decisions. How can the Census Bureau ensure that people are aware of logistics, details, and support options for their surveys? How can they address concerns around legitimacy? Our team of Harvard students is working with the Census Bureau to help them find an answer.

Understanding the People

We approached these questions from several angles. We looked into existing research from the Census Bureau about attitudes toward the 2020 Census and connected with five experts to learn more about the inner workings of the Bureau. We also spoke to two field representatives, whose job is to knock on selected residents’ doors to administer in-person surveys. They shared their perspective on how and why people do and don’t answer surveys:

• A major challenge that field representatives face is silent refusal. People may set up meetings with the representatives, but then they don’t show up nor respond to follow-up communication. This response is more common than a hard no, and it wastes time and resources.

• Field representatives may have difficulty accessing respondents at their homes in certain neighborhoods, such as apartment complexes, gated communities, and rural households.

Most importantly, we reached out to potential respondents—since every U.S. resident could someday receive a survey, we have a very broad user base! We interviewed 27 people across the Cambridge area, learning from their reactions to our simulation of being selected for a survey, as well as their previous experiences with filling out the decennial census or other demographic surveys. To ensure we heard from people with a diversity of backgrounds, varying in gender, age, race, education level, and employment status, we visited transit centers, convenience stores, laundromats, and public community centers, as well as stopping people on the street.

Demographic overview of our interview sample

While every person we spoke with had a different and nuanced experience, when we looked over the interviews together, we discovered the following common concerns:

• Legitimacy of the survey, process, and interviewer is a major concern of census respondents.

• Lack of awareness of the purpose of surveys or the Census Bureau’s work often leads respondents to be disengaged from the survey process.

• Respondents are worried about privacy and potential data breaches that may result in misuse of personal information.

• Mood and competing commitments decrease the respondents’ likelihood of answering a survey.

• Growing distrust in democratic systems deter respondents from taking part in government-related events, including surveys conducted by the Census Bureau.

Next Steps

From here, we’ll brainstorm and prototype solutions to address the needs that we uncovered from our initial research. Then, we’ll talk to more people like Augusto and test to what extent our prototypes improve their experience with household surveys, using their input to provide final recommendations to the Census Bureau. We can’t wait to continue our work!

Alison Chen, Jessica Nunez, Mahesh CR, Tiffany Yu, & Vishnu Rajeev

Harvard Students Partner with U.S. Census Bureau to Address Declining Response Rates

A few years back, the Mormon Church announced that it was ending its door-to-door missionary work. Some church members attributed the change in their decades-old strategy to a lack of trust among people to open their doors to strangers. But it’s not the only organization struggling with trust and legitimacy that sends representatives to your doorstep.

The U.S. Census Bureau, part of the U.S. Department of Commerce, counts the resident population in the United States every ten years, regardless of citizenship or legal status. The upcoming 2020 Census will also include a new online option, in addition to the traditional mail-in option and Census enumerators knocking on doors.

A census enumerator and a respondent during a census survey (source: U.S. Census website)

You might not be aware that the Census Bureau conducts over 100 additional surveys, such as the American Community Survey, the Economic Census, and the Current Population Survey. The data collected from these surveys helps allocate federal funds every year. They allow states, local communities, and businesses to make informed decisions about supporting populations in need, investing in schools, hospitals, and roads, and growing businesses.

A listing of major U.S. Census Bureau surveys (source: U.S. Census website)

Depending on the survey, the Census Bureau selects addresses that are representative of the targeted population. Trained Census Bureau field representatives then typically administer surveys in person or over the phone.

Unfortunately, the response rate to Census surveys has decreased by more than ten percent in the last decade. Additionally, field representatives report a trend of growing mistrust and skepticism among respondents.

Lisa Clement, a senior survey director at the Census Bureau, explains, “This trend might be the result of a lot of factors, such as increasing distrust of the government, privacy concerns, data security, and doubts about the credibility of the surveys. The number of households refusing to participate has almost doubled in the past ten years.”

More broadly, the Census Bureau is wrestling with the following questions:

• How can we address the concerns of Census respondents and increase the number of participants from surveys?

• Can we leverage advances in technology and design to address these issues?

Introducing the Harvard Team

We are a team of five students across Harvard University enrolled in DPI-663, a field class on Technology and Innovation in Government taught by Professor Nick Sinai. We bring together diverse experiences and skills, ranging from software engineering to product management to strategy consulting. Here is more about us:

Alison Chen is a first-year student at Harvard College passionate about the intersection of technology and public service. She is exploring the social sciences and applied math, with interests in public policy, tech, and philosophy. She is particularly excited to work with the Census this term to find new ways to innovate within government to benefit all citizens.

Jessica Nunez is a joint degree MBA at the Tuck School of Business at Dartmouth and an MPA at the Harvard Kennedy School.  Previously she worked for two years as a 1st-grade bilingual teacher with Teach For America and Chicago Public Schools. Additionally, she has five years of experience as a consultant at Deloitte and Touche. She is passionate about closing the student achievement gap through education technology.

Mahesh CR is a second year MPA student at Harvard Kennedy School. He started a technology-based assessment immediately after his engineering degree. He did his MBA in India and worked as a consultant with EY on projects such as Digital Bangladesh and Skill India. He interned with McKinsey and will be joining the firm full time after graduating from HKS. He is interested in the intersection of technology and government.

Tiffany Yu is a senior at Harvard College majoring in Computer Science with a minor in Psychology. She is interested in how technology creates both new challenges and new avenues for innovation within the government.

Vishnu Rajeev is an MPA candidate at the Harvard Kennedy School. He has more than five years of work experience in strategy consulting, public policy analysis, and venture capital. Previously, he has worked with Monitor Deloitte and FSG in their social innovation platform exploring technologies and startups in emerging markets such as India and East Africa. He is passionate about technology and how it influences business and policy.

Next Steps

For the 2019 spring semester, we’ll be working closely with Census officials—including Lisa Clement, a senior survey director, Ruth Chan, a respondent advocate, and many others.

Our first step is to interview people in the Boston area, in order to learn about how they feel about Census Bureau surveys and responding to surveys more generally.  Later in the semester, we’ll brainstorm ideas, develop and test a few of them, and then provide broader recommendations to the Census Bureau.

The Census Bureau is the premier statistical agency that provides the foundation of data-driven policymaking in the United States. Unlike the Mormon church, the Census Bureau is not going to—and should not—shut-down its door-to-door engagement. But if the Census Bureau can better understand people and their attitudes and behaviors towards surveys, perhaps they can slow or even reverse the declining response rate.  We’re excited to help!

If you have ideas or want to talk about your experiences with Census surveys, drop us an email at vir837@student.hks.harvard.edu.

Alison Chen, Jessica Nunez, Mahesh CR, Tiffany Yu, & Vishnu Rajeev