A Lesson in Rejection: Researching the Household Survey Experience

Augusto is 58 years old. He moved to the United States in 2008, and he isn’t very comfortable with English. When asked whether or not he would respond to a mailed request to complete a survey conducted by the U.S. Census Bureau, he said, “I have to be careful. I need to understand the question and what it contributes to and what improvement is needed.”

There are many people like Augusto in the United States who have reservations about answering these demographic surveys. However, the Census Bureau needs their information to provide up-to-date statistics about the American population to allow organizations to make informed community decisions. How can the Census Bureau ensure that people are aware of logistics, details, and support options for their surveys? How can they address concerns around legitimacy? Our team of Harvard students is working with the Census Bureau to help them find an answer.

Understanding the People

We approached these questions from several angles. We looked into existing research from the Census Bureau about attitudes toward the 2020 Census and connected with five experts to learn more about the inner workings of the Bureau. We also spoke to two field representatives, whose job is to knock on selected residents’ doors to administer in-person surveys. They shared their perspective on how and why people do and don’t answer surveys:

  • major challenge that field representatives face is silent refusal. People may set up meetings with the representatives, but then they don’t show up nor respond to follow-up communication. This response is more common than a hard no, and it wastes time and resources.

  • Field representatives may have difficulty accessing respondents at their homes in certain neighborhoods, such as apartment complexes, gated communities, and rural households.

Most importantly, we reached out to potential respondents—since every U.S. resident could someday receive a survey, we have a very broad user base! We interviewed 27 people across the Cambridge area, learning from their reactions to our simulation of being selected for a survey, as well as their previous experiences with filling out the decennial census or other demographic surveys. To ensure we heard from people with a diversity of backgrounds, varying in gender, age, race, education level, and employment status, we visited transit centers, convenience stores, laundromats, and public community centers, as well as stopping people on the street.

 
Demographic overview of our interview sample

Demographic overview of our interview sample

 

While every person we spoke with had a different and nuanced experience, when we looked over the interviews together, we discovered the following common concerns:

  • Legitimacy of the survey, process, and interviewer is a major concern of census respondents.

  • Lack of awareness of the purpose of surveys or the Census Bureau’s work often leads respondents to be disengaged from the survey process.

  • Respondents are worried about privacy and potential data breaches that may result in misuse of personal information.

  • Mood and competing commitments decrease the respondents’ likelihood of answering a survey.

  • Growing distrust in democratic systems deter respondents from taking part in government-related events, including surveys conducted by the Census Bureau.

Next Steps

From here, we’ll brainstorm and prototype solutions to address the needs that we uncovered from our initial research. Then, we’ll talk to more people like Augusto and test to what extent our prototypes improve their experience with household surveys, using their input to provide final recommendations to the Census Bureau. We can’t wait to continue our work!

Alison Chen, Jessica Nunez, Mahesh CR, Tiffany Yu, & Vishnu Rajeev

Harvard Students Partner with U.S. Census Bureau to Address Declining Response Rates

A few years back, the Mormon Church announced that it was ending its door-to-door missionary work. Some church members attributed the change in their decades-old strategy to a lack of trust among people to open their doors to strangers. But it’s not the only organization struggling with trust and legitimacy that sends representatives to your doorstep.

The U.S. Census Bureau, part of the U.S. Department of Commerce, counts the resident population in the United States every ten years, regardless of citizenship or legal status. The upcoming 2020 Census will also include a new online option, in addition to the traditional mail-in option and Census enumerators knocking on doors.

A census enumerator and a respondent during a census survey (source: U.S. Census website)

A census enumerator and a respondent during a census survey (source: U.S. Census website)

You might not be aware that the Census Bureau conducts over 100 additional surveys, such as the American Community Survey, the Economic Census, and the Current Population Survey. The data collected from these surveys helps allocate federal funds every year. They allow states, local communities, and businesses to make informed decisions about supporting populations in need, investing in schools, hospitals, and roads, and growing businesses.

A listing of major U.S. Census Bureau surveys (source: U.S. Census website)

A listing of major U.S. Census Bureau surveys (source: U.S. Census website)

Depending on the survey, the Census Bureau selects addresses that are representative of the targeted population. Trained Census Bureau field representatives then typically administer surveys in person or over the phone.

Unfortunately, the response rate to Census surveys has decreased by more than ten percent in the last decade. Additionally, field representatives report a trend of growing mistrust and skepticism among respondents.

Lisa Clement, a senior survey director at the Census Bureau, explains, “This trend might be the result of a lot of factors, such as increasing distrust of the government, privacy concerns, data security, and doubts about the credibility of the surveys. The number of households refusing to participate has almost doubled in the past ten years.”

More broadly, the Census Bureau is wrestling with the following questions:

  • How can we address the concerns of Census respondents and increase the number of participants from surveys?

  • Can we leverage advances in technology and design to address these issues?

Introducing the Harvard Team

We are a team of five students across Harvard University enrolled in DPI-663, a field class on Technology and Innovation in Government taught by Professor Nick Sinai. We bring together diverse experiences and skills, ranging from software engineering to product management to strategy consulting. Here is more about us:

Alison Chen is a first-year student at Harvard College passionate about the intersection of technology and public service. She is exploring the social sciences and applied math, with interests in public policy, tech, and philosophy. She is particularly excited to work with the Census this term to find new ways to innovate within government to benefit all citizens.

Jessica Nunez is a joint degree MBA at the Tuck School of Business at Dartmouth and an MPA at the Harvard Kennedy School.  Previously she worked for two years as a 1st-grade bilingual teacher with Teach For America and Chicago Public Schools. Additionally, she has five years of experience as a consultant at Deloitte and Touche. She is passionate about closing the student achievement gap through education technology.

Mahesh CR is a second year MPA student at Harvard Kennedy School. He started a technology-based assessment immediately after his engineering degree. He did his MBA in India and worked as a consultant with EY on projects such as Digital Bangladesh and Skill India. He interned with McKinsey and will be joining the firm full time after graduating from HKS. He is interested in the intersection of technology and government.

Tiffany Yu is a senior at Harvard College majoring in Computer Science with a minor in Psychology. She is interested in how technology creates both new challenges and new avenues for innovation within the government.

Vishnu Rajeev is an MPA candidate at the Harvard Kennedy School. He has more than five years of work experience in strategy consulting, public policy analysis, and venture capital. Previously, he has worked with Monitor Deloitte and FSG in their social innovation platform exploring technologies and startups in emerging markets such as India and East Africa. He is passionate about technology and how it influences business and policy.

Next Steps

For the 2019 spring semester, we’ll be working closely with Census officials—including Lisa Clement, a senior survey director, Ruth Chan, a respondent advocate, and many others.

Our first step is to interview people in the Boston area, in order to learn about how they feel about Census Bureau surveys and responding to surveys more generally.  Later in the semester, we’ll brainstorm ideas, develop and test a few of them, and then provide broader recommendations to the Census Bureau.

The Census Bureau is the premier statistical agency that provides the foundation of data-driven policymaking in the United States. Unlike the Mormon church, the Census Bureau is not going to—and should not—shut-down its door-to-door engagement. But if the Census Bureau can better understand people and their attitudes and behaviors towards surveys, perhaps they can slow or even reverse the declining response rate.  We’re excited to help!

If you have ideas or want to talk about your experiences with Census surveys, drop us an email at vir837@student.hks.harvard.edu.

Alison Chen, Jessica Nunez, Mahesh CR, Tiffany Yu, & Vishnu Rajeev