Tips on surveying
Clearly it would take too much time and money to survey everybody who has money to spend in your community. The trick is to design your survey so that you capture enough information to make it meaningful, while still making it a manageable and enjoyable experience. Remember that too much information will be confusing and make decision-making even more difficult. There are sections below on who to survey, designing the questionnaire and carrying out the survey.
Designing the Survey
When designing your questionnaire, it is important to have first spent some time deciding exactly what information you want to find out and then thinking about the best way to ask the questions. Time spent carefully planning your questions at the start will prove invaluable later. The most important point is to remember how you are going to use the results, because there is no point in having a huge amount of indigestible data. So do think through very carefully how you are planning to analyse the data you receive and bear this in mind when creating your questionnaire. Here are some additional ideas to get you started.
Some tips for designing the survey form:
- Where possible, it is useful to use what is known as 'a 5-point Belbin scale' (e.g. 'Very good, good, don't know, bad, very bad'). Five options are manageable but provide a variety of possible responses.
- Include 'don't know' - it's ok if people don't!
- Don't have prompts for people - if you're asking a 'why' question, let people tell you why in their own words. You can record it against several popular categories afterwards
- A 'quantitative' figure such as 'x per cent said they sourced their window cleaner from outside their area', is best supplemented with a 'qualitative' understanding of why a local window cleaner isn't used - this will help you to know how to plug such a leak - should you publicise the existing local window cleaners, develop their skills, or help a local person set themselves up in this role.
- When local people do face-to-face interviews, conversations often happen about their feelings and experiences that are invaluable in understanding an area. These less formal outcomes of the survey might be just as important as the actual answers to the questions, so you need to allow a space for interesting information to be imparted.
- Choose a time of day when businesses are less busy - Monday early afternoon is often a quiet time for shops. Early January is even better!
- Decide whether cold calling or a pre-arranged interview is better. Small shops often respond better to cold calling - when you turn up in the shop and meet them face-to-face, they can decide there and then if they trust you. Arranging an interview can sound too formal or complicated and be off-putting. Larger shops often prefer a pre-arranged time to talk, and this will make them take it more seriously.
- Start by introducing yourself and the project.
- It is really helpful to have and to show identification, for example a letter naming the person as an interviewer for the project, and a telephone number for anyone with any qualms to ring. If you have a leaflet about the project, offer it.
- Before they ask the questions they should emphasise, first, confidentiality - names of interviewees will not be used in any report, and, second, what feedback interviewees will receive.
- They should be able to explain what the project is, and what the survey is for, clearly and concisely.
- They should be sensitive to the position of the interviewee. For instance, they should not push for long supplementary answers if someone is starting to look bored.
Initial planning of feedback
At the same time as designing the survey, you'll want to think about how you are going to feed back the results of the survey to the people who have been interviewed. Perhaps you could line up an article in the local paper for a certain date, or arrange to have a display in a village hall or community centre. Deciding at least one communication channel in advance means that you can tell people who are interviewed where they can find the results, even though you will probably want to decide how to do the bulk of the communication once you have seen the results.
It's important to make sure that interviewers feel safe when interviewing either in the streets or door-to-door. Some people might feel safe going out alone, while others might prefer to go out in pairs - particularly in the evening.
About young people
It is not easy to include the views of young people outside of a group situation, as it is illegal to ask children under 16 for their views without written permission from their parents. So you can't stop children as part of a street survey. You can, however, ask children in school, as teachers are legally considered guardians of children at that time, or during a door-to-door session as long as the parents give permission.
Young people can make enthusiastic and reliable researchers. One idea might be to get young people to interview and be interviewed by other young people on their spending patterns. This would be a fun way to involve school-aged children in your study. The mapping process would also link well with several parts of the national curriculum, for example mathematics, geography and citizenship.
Who to Survey
Choose your sample
To be reasonably sure that the sample will give useful results, you must 1) use a large enough sample and 2) ensure that the sample of people you survey are similar, as a group, to the entire group of people you want to survey. But what does 'similar' mean? Must they support the same football teams or eat the same food? You will need to decide which are the most important personal factors that may affect people's spending patterns. For resident surveys these might include gender; age; social background; ethnicity; or disability status. Alternatively, it might be occupation or even location of home. For business surveys it might be the size of the business or the nature of the business that has most impact on its spending patterns. For the public sector budget-holders it might be easiest to approach it by sector (eg schools, housing, procurement, and so on) and to really focus on the bigger budget holders.
Do make sure that you get an adequate sample from groups that are hard to reach. Examples of these are 18 - 24 year olds, because they don't like surveys, or full-time workers, because they aren't around their local area during the day - so try to do some surveying in the evening too.
The role of the person co-ordinating the surveys is crucial in making sure that this happens. It is very easy, particularly where lots of people are helping with the survey, for everyone to be so concerned with getting any surveys at all that they forget about seeking representativeness.
How many people?
The greater the sample size, the greater the accuracy of the results and the range of interesting leak-plugging ideas. Also, the more people you interview the more people who now know about the leak-plugging work you are exploring. It is important to have a good idea of what people think, but also to be realistic about the number of people it is possible to survey. Do realize that with leak-plugging it is not necessary to acquire accuracy - if a large number of people say that for them a particular leak exists then it is still worth pursuing. We therefore do not suggest particular sample sizes for you to achieve - simply aim for as many as you can with the willing support of the volunteers, staff members or researchers your resources can mobilise.
When to Survey
Once people start surveying they usually see the value of it, but getting started is hard! For consumer surveys, remember that people prefer standing outside and knocking on doors in summer. For business or public service budget-holder surveys you would be best to do the survey in their office so that they have the data to hand. Try to avoid busy times of year, such as just before the end of the financial year when all the budget setting is taking place.
How to Survey Residents
For a street poll, choose places where different sorts of people gather. The interviews should be done at different times of day, and with a balance of weekend and weekdays, in order to catch different sorts of people; also choose the locations carefully to maximise coverage of the area concerned.
Check with local residents that people feel comfortable opening doors to strangers - in some estates we found that people only open the door to the council. It is important to have identification visible.
Sometimes surveys can be conducted by telephone. However, telephone surveys do not tend to lead to as much interaction between interviewer and interviewee. The personal contact of face-to-face interviews led to a lot of additional stories and background knowledge being passed on. It also generated a lot of interest in Plugging the Leaks. If you want to do a large number of surveys however, telephone surveys are a good way of getting a random spread of households across an area, but it will obviously only cover people who have a telephone and are not ex-directory.
How to Survey Businesses
On the whole, businesses respond better to a quick chat with the interviewer filling in the survey form with them, rather than getting a survey form in the post. There are exceptions to this - in Milford Haven, one businesswoman filled in her postal survey form completely with comments in the margins, and was obviously keen for someone to hear her views. Generally, postal surveys elicit a 5 - 10% response rate, whereas personal contact elicits a 50-70% positive response. So unless you have a very large number of businesses to survey, personal contact will be more effective.
In Harehills, the local partner visited businesses in Harehills Lane to ask if they would help him complete the survey questions. In one afternoon, he spent two hours visiting 10 shops. Of these, 5 were happy to talk and have the survey form filled in, 2 were happy to talk informally but didn't want to use the survey form, and 3 weren't interested at all. The partner gained a much deeper insight into local business realities by actually talking to people than by reading forms. In some businesses, 'it was like sitting with the elders - they would invite me in the back and all the uncles and friends would be there, and we sat round discussing the local economy'.
If local people are to do the interviews, it is important to give them at least a brief training session. The key points to make to them are:
They should then have the opportunity to practice, initially under supervision so that you can provide feedback.
Collecting stories as well
People will often come up with stories or snippets of information while they're answering the survey. These are invaluable, so be sure to record them as soon as possible. Make a note next to the story of which question it refers to most. They'll be really useful when you come to try and understand the results afterwards, and will help you to tell the whole story of your area. The stories can be used anonymously - a girl in the estate said that. If you know who the person was and you want to use the story with their name, you'll have to seek permission, as the questionnaire process is strictly confidential. A group debrief session for all of the interviewers after the surveys are finished is a good way to share and collect the 'soft' information like stories that people collect during the interview process.