For Education & Nonprofit

FAQs

1. What experience do you have working with non-profits and educational institutions?
We specialize in education and non-profit marketing research, closely following industry trends, and have worked with a wide variety of independent schools, colleges, foundations, arts & culture organizations, and other organizations conducting qualitative and quantitative studies with every layer of the industry, including with CEOs, development directors, board members, donors, students, parents, alums, and the general public.
2. Can research help me make decisions, given my organization’s situation? What methodology would you recommend?
We are happy to talk with you about the merits of research for your organization and what methodology would be most appropriate. Research is appropriate for most of the organizations that contact us when they have key decisions to make, and need to understand the perspectives of major stakeholders, such as donors or students. There are cases, however, in which research isn’t appropriate. Our goal is to provide resources to help our clients make better decisions, and if research is not the tool needed, we certainly would advise against it.
3. I think I have a good sense of my organization’s position in the market, but I can’t get buy-in from the C-suite. Can research help?
One of the key benefits of doing marketing research is that the findings give key decision-makers an unbiased understanding of the organization’s position in the market, which enables them to make data-based decisions. We often find that people we work with have a general understanding of the direction that the organization should go in, and the results of a marketing research study solidify their assumptions and help them get buy-in from the C-suite.
4. How long does the research process take? When can I expect results?
The length of a study depends on the methodology, but most studies take 6 to 8 weeks. For clients with tight deadlines, we can work together to ensure those needs are met. After learning about your specific research goals and needs, we can provide you with a time line for your project.
5. Who in my organization should be involved in the research process?

It depends. If your organization needs to make a major decision or there is a great deal of disagreement around the decision, it is critical to include key decision-makers early in the research process. If key decision-makers agree on the methodology and the research instrument (e.g., questionnaire or moderator guide), they are more likely to accept the research results and less likely to attack the process when the research is complete.

We’re happy to talk with you about who you would want to include in the research process, given the factors at play in your organization.

6. As an organization with limited resources, our budgets can often be tight. What does a typical study cost?
The cost of a study varies depending upon the methodology. After speaking with you about your research needs, we can usually provide a budget with 2 to 3 business days.
7. Can I trust the results from a survey? It seems like political surveys always get it wrong.
Yes, you can still trust quantitative surveys. We have consistently found that when we ask questions in surveys that can be verified by external data, the results are highly comparable. Recently, there has been more skepticism about survey findings, with many people believing that the polling for the 2016 Presidential election was inaccurate. The surveys, however, were highly accurate: The polls indicated that Hillary Clinton would win the popular vote by 3% and she won by 2%. Pew Research Center, which is a highly respected research organization, has an interesting video about the accuracy of surveys, looking at the accuracy of survey findings from a number of different perspectives: http://www.pewresearch.org/fact-tank/2018/05/14/can-we-still-trust-polls/
8. What’s the best way to survey consumers, now that less than half the population has landlines?

The best way to survey consumers is online, using companies with panels of consumers who have agreed to participate in surveys. The panels we use have millions of consumers on them, and it’s possible to target the respondents by age, income, or other characteristics.

While online surveys are ideal, they aren’t always feasible, particularly when surveying market areas with relatively small populations. In those cases, we may do a combination online-phone survey or a survey entirely by phone. We have consistently found that when conducting a combination study, the results from the phone and online components rarely differ significantly from each other, which gives us confidence in these approaches.

We have experimented with surveying people on their cell phones but have found that most people are unlikely to answer a call from an unknown caller on their cell phones. When they do answer, they could be anywhere—driving in their cars, on vacation, or shopping. The number of questionnaires we can complete by cell phone is generally not worth the added cost.

9. What sample size should I have in order to have confidence in the findings?

The first consideration in deciding on a sample size depends on how you define your market area. If you make decisions based on your entire market area, you only need to ensure your sample size accurately reflects the whole. If you have multiple areas or want to divide your total area into regions, you would need to have a sufficient sample size in each region in order to be able to draw statistically accurate conclusions by region.

The second consideration relates to the error rate. The error rate is a calculation based on the sample size, the size of the population you are sampling from, and the degree of confidence you set. For our studies, we set the confidence level at 95% (we are 95% confident that the error rate is what we say it is.)

In terms of the size of the population, because of a statistical property called the law of large numbers, the error rate for a specific sample size is stable once the size of the underlying population is greater than 99,999. For example, the error rate for a population base of 100,000 is exactly the same as that for a population base of 1 million.

In terms of recommending a specific sample size, larger the sample size the lower the error rate, but the more the study will cost. We work with our clients to decide how much error they can tolerate relative to the constraints of their budget. Typically, we recommend a sample size of 300, which has an error rate of ±3.4-5.7 percentage points. We believe this size is the best combination for highly accurate results while keeping costs low.

Another aspect of deciding on a sample size is the credibility of the data to key stakeholders. There are situations in which clients want a larger sample size to give more confidence that they are moving in the right direction. If you are a political candidate, the difference between getting 50.1% of the vote and 49.9% of the vote is huge, so you might want a larger sample size to reduce your error rate. For most of our clients, that extra percentage point or two in accuracy is not worth the additional investment. The decisions they’d make would be no different whether their name recognition, for example, was 25% vs. 28%. Finally, budget is always a consideration. We work with our clients to design the best study we can within the constraints of their budgets.

10. How many questions can I ask in a survey?

We recommend limiting questionnaire that can be completed in 15 minutes or less. Research that shows that people can be kept fully engaged in a survey for up to 20 minutes, but that the quality of people’s responses deteriorates after that point.

In general, people can complete 4 to 5 questions in a minute, which means we can ask about 60 to 75 questions in 15 minutes. The number of questions that can be completed in a minute varies, though, depending on the length and complexity of the question and the answer categories.

11. Should I do in-person or on-line focus groups?

Deciding between doing focus groups in-person vs. online depends on a variety of factors. The first factor relates to the characteristics of the people you want to participate in the focus group. People with complex or busy schedules, such as physicians, have an easier time committing to an online focus group. We generally hold our online focus groups over 3 days, with participants committing to participating at least 30 minutes a day, but can participate at any time that is convenient for them based on their schedule.

Also, if the potential participants live in a wide geographic area, we’d recommend an online group. People don’t like to travel more than 15 to 20 minutes to attend a focus group, which limits your potential participants for an in-person group.

On the other hand, if the goal of the focus group is to explore a new concept or to talk about a subject that participants don’t know much about, an in-person group would be more effective, because the moderator can create a conversation by probing people’s responses and participants can gain insight into their own perceptions and attitudes by hearing from other people in the group. In addition, the moderator has the flexibility in an in-person group to rearrange the guide or go in a different direction based on the responses. With an online group, the moderator can post follow-up probes to people’s responses, but their responses are not always immediate. In addition, the moderator guide is generally fixed in advance, and while some modifications can be made to the questions asked in Day 2 or 3, an online group has less flexibility than an in-person group.

12. Can I get a Net Promoter Score? How does that affect the budget?

We regularly include the question about recommending our clients and their key competitors that we use to calculate the Net Promoter Score (NPS). The NPS question is, “One a scale from 0 to 10, how likely are you to recommend COMPANY to a friend?” People who give a rating of 0 to 6 are Detractors, those who give a 7 to 8 are Passives, and those who give a 9 or 10 are Promoters. The NPS is calculated by subtracting the percentage of Detractors from the percentage of Promoters.

Including the NPS question and analysis does not affect the budget for a study.

13. When should I do a qualitative study versus a quantitative study?

In general, a qualitative study is appropriate when you are trying to get in-depth information about how people make decisions and/or why they have the attitudes or perceptions they have. Qualitative research is also valuable when you want information about a new concept, product, or service—particularly one that’s unique, is unlike something people have experience with, or that is complex and challenging to understand.

A quantitative study is appropriate when you need to know what proportion of your target market has specific attitudes, beliefs, and preferences and when you need to be able to track changes in those attitudes, beliefs, and preferences over time.

There are many cases in which the ideal methodology would be to conduct both qualitative and quantitative research. The understanding we’d gain about the topic from the qualitative research can help inform the questions we ask in the quantitative research. Sometimes we start with the quantitative research and follow-up with the qualitative research to get at the “whys” behind the quantitative findings.

If both types of research would be helpful, but there isn’t room in the budget, we will work with the client to determine which approach would be most useful If qualitative research as most appropriate, we are confident that clients can act on the basis of qualitative research alone. With over 40 years of experience in conducting qualitative followed by quantitative research, we’ve found that the themes from the qualitative research are confirmed by the quantitative research. We wrote a white paper comparing the finding from qualitative and quantitative research, which demonstrated the comparability of the findings from the two methodologies: http://marketstreetresearch.com/wp-content/uploads/2014/09/Trusting-The-Room.pdf

14. What protocols do you follow in order to protect the confidentiality of our donors/students/clients/etc.?

Market Street regularly conducts surveys using lists provided by our clients, such as lists of students, donors, board members, parents, and alumni. We have developed protocols employ those protocols with every list provided to us by a client.

We’d be happy to provide you with the specific protocols we follow in order to protect the confidentiality of any list you provide us.