3B: Surveying Visitor Satisfaction

Why monitor visitor satisfaction at a destination?

As well as highlighting a destination’s strengths and the areas requiring improvement, a visitor satisfaction survey can provide a sound basis for monitoring performance and for future marketing, infrastructure and business development. Some of the ways it can achieve this (depending on the questions asked, the people spoken to and the type of research carried out) are by:

• identifying the profile and origin of visitors
• examining their behaviour once at the destination
• evaluating the effectiveness of marketing activities
• assessing the quality of facilities, services and levels of visitor satisfaction
• testing reaction to new or proposed developments, services or promotional materials.

Carrying out visitor satisfaction surveys should not be seen as a one-off exercise. It is important to carry out these surveys on a regular basis (ideally every one or two years), to build up an ongoing and comparable picture of visitor satisfaction with a destination. More importantly, this will allow visitor reaction to any changes made at the destination to be monitored, and trends to be identified.

Tourism South East offers a ‘Destination Benchmarking Service’ which enables destinations to carry out a standardised visitor satisfaction survey allowing them to compare their survey results over time, as well as with other destinations on a local, regional or national basis. Contact Tourism South East Research Services for more information.

How to conduct a visitor satisfaction survey: the 14-point plan

The following section presents a 14-point plan for conducting and managing a visitor satisfaction survey. The aim is to conduct a survey to produce credible, usable and comparable research and avoid many of the common pitfalls of carrying out market research.

However, depending on the resources and budget available it may be preferable to commission a professional market research agency to carry out the survey.

The following will help if outsourcing is considered:

• helpful advice and, in some cases, research services can be obtained from the Regional Development Agency or tourist board and eventually through the English Tourism Intelligence Partnership (ETIP)
• for a list of market research agencies registered with The Market Research Society search their online Research Buyers Guide at www.rbg.org.uk
• for general information about market research, search The Market Research Society website www.mrs.org.uk

If commissioning research from an agency is too expensive, the choice is to either commission elements of the process (such as the interviewing or analysis) or to try to do the work in-house. The 14-point plan, given below, sets out the steps that need to be followed in carrying out visitor satisfaction surveys. It will help whether the decision is to:

• commission a full survey
• commission certain elements of the process
• do the work in-house.

1. Creating a survey plan

Before embarking on a market research project it is worth taking some time to think about what the aims of the project are. At this stage it is not necessary to think about the phrasing of the actual questions themselves – this will come later.

The survey plan is actually a list of the answers needed from the research, rather than a list of questions. Consider some of the following areas and think about the services and facilities in the destination before going any further:

• Think about the type of information needed and the reason for carrying out the research. Is there a particular issue in the destination that requires specific attention, eg parking facilities, facilities for children, signage, shop opening times or standards of street cleanliness?
• Surveys can measure both negative and positive visitor reactions. Is there something that works well in the destination? What do visitors think about this? What about recently implemented projects, eg a new bus service or a new shopping area – are these useful for visitors?
• Who will be looking at this research? What are they expecting a report on, eg is there a need to build a case for funding, or perhaps gather evidence for a new facility, or a change to existing services?
• Talk to colleagues to gather their ideas. If possible, look at examples of visitor satisfaction surveys and reports from other destinations. More importantly look at the model formats for surveys suggested in this service particularly if the data is to be shared and compared with others. (See 3A: Measuring Overall Tourism Performance for more information on performance data collection and comparison with others.)
• Will this survey be repeated to build up visitor trends over time?
• What basic information about the visitor is required? Consider elements like age, transport used, length of trip, frequency of visit to destination, area of region or country lived in, number of people in the party.

Once it has been decided the research should achieve, it is time to go on to the next stage in the plan.

2. Deciding on the most appropriate survey type

The second stage is to think about the most appropriate survey type (or types) to use. There are two types of market research:

• Quantitative research - Collecting information from respondents via a set of questions on a questionnaire. Responses can be collected either through an interviewer talking directly to the respondents (either face to face or over the telephone) or by respondents being sent a questionnaire in the post or picking one up themselves, then completing and returning it.
• Qualitative research - Talking to an individual or group of people for about an hour to gather their opinions. This type of research is structured and may contain deliberate questions, but the overall pattern would be a discussion rather than gathering answers to a set questionnaire. The success of this method depends very much on the skill of the interviewer.

Generally, quantitative research is used for visitor satisfaction surveys – usually face-to-face interviews conducted by an interviewer. This method of research has the following advantages:

• Personal contact with the respondent increases the number of interviews achieved (response rate).
• it is easier to control who the interviewer talks to, and therefore talk to a more representative group of people.
• Verbal explanation can be given by the interviewer if necessary, and the questions asked can therefore be slightly more complex.
• Misunderstandings can be reduced as the interviewer can gauge how the respondent has interpreted a question and clarify if necessary.
• Unhelpful or inadequate replies from the respondent can be probed by the interviewer.
• Missing answers (blanks) are generally avoided. Few people refuse to answer questions put to them, but may leave them blank when filling in the questionnaire themselves.
• There is control over when the respondent is approached, ie during their visit when their experience is still fresh in their memory rather than two weeks after a visit when they receive a questionnaire from you in the post.
• This method allows a considerable amount of information to be collected quickly and accurately by well-trained staff.

There are a couple of disadvantages:

• This type of survey can be subject to interviewer bias, such as the interviewer not selecting the respondent randomly. Trained staff and strict sampling (see below) can minimise this problem.
• This type of research can be costly to implement.

3. The sampling method and size

Organising a questionnaire survey involves statistics and dealing with samples of population. In this case the population is all the people who visit the destination and the sample is those who are selected for interview. The accuracy of the data collected, and therefore the conclusions that can be drawn from the research, will depend on how the sample is selected.

A basic knowledge of sample design is necessary to plan a survey and the following notes should help with this. However, it may be useful to seek help on sample design from research experts in the area, eg the RDAs, DMOs or Tourist Boards.

Basically, sample design looks at WHO, WHEN and WHERE:

• WHO you interview
• WHEN the interview takes place
• WHERE the interview takes place.

A sample framework needs to be created to ensure that a representative sample of visitors is interviewed during the survey. The size of destination and the depth to which the data will be analysed will determine the overall number of people interviewed (the sample size). Generally, to get accurate results from a survey, aim to interview a minimum of 400 people.

To achieve a representative sample (by talking to a cross-section of visitors) break down the visitors into groups according to:

• day visitors to the destination
• overnight visitors to the destination
• a good cross-section of age groups (this is usually controlled with bands of ages for example 16-24, 25-34, 35-44, 45-54, 55-64, and 65 and over, or more)
• male or female visitors
• area of residence, eg visitors living within a radius of 20 miles from the destination and visitors living more than 20 miles from the destination
• method of transport used to travel to destination.

A cross-section of visitors should be interviewed at different times of day on each day of the week (including the weekend) to ensure the sample is representative. Finally, make sure that interviewers are positioned at a variety of places within the destination, ideally at places used or passed by visitors, and that interviews are carried out throughout the day.

A sample framework for the survey can be produced by working out which visitor sub-groups are to be interviewed and how many people to interview in these sub-groups. Each interviewer working on the survey will be instructed to interview a number of people from each of the sub-groups. This is known as "quota sampling". The interviewer’s aim is to fill each quota. Once a quota has been filled, any subsequent respondent from that sub-group can be thanked for their assistance but will not need to be interviewed at length.

4. Looking at the resources needed

At this point in the plan it is important to look at the resources available in terms of finance, time and human resources to make sure that the plans are not too ambitious. If this exercise is not carried out, then the survey can quickly run out of control and take up more resources than expected.

The main cost is likely to be staff time, not just in terms of interviewing, but also in:

• planning and preparing the survey
• training interviewers if they are not experienced
• carrying out the analysis work.

The total number of work hours needed to complete the project will need to be calculated. How much money is spent on the survey will, of course, depend on the amount and nature of the information to be collected. Before finalising the calculations, consider whether all the necessary skills for the project exist within the organisation or whether expertise or additional help will need to be bought in from outside (for example to do the interviews, design the questionnaire or analyse the results).

5. Analysis methods

Before writing a questionnaire it is important to think about the method to be used to analyse the results of the survey. Again, this is an area where it is useful to seek help from research experts. Generally, surveys are analysed with the help of a computer software package. Analysing the survey data involves inputting the information collected on the questionnaires into the package so that the answers are aggregated together to produce a set of data.

There are two main types of software that can be used for this task:

• statistical packages, eg SPSS, SNAP
• spreadsheet packages, eg Microsoft Excel.

Inputting the information from the questionnaire involves "coding" the answers that the respondent has given. This simply means that answers are converted to a single digit code. Coding is often printed on the questionnaires and takes the form of a list of possible responses to a question with a number alongside each response. For example, the coding for a simple question which required either a "YES" or "NO" response would be YES=1 and NO=2.

If the respondent answered "YES" then the interviewer would tick box 1 and the person entering the data from the questionnaire into the software package would type "1" into the computer.

Sometimes questions can bring more than one response. For example, if a respondent was asked which days of the week they usually visit the shopping centre at the destination they could answer anything from "never" to "every day of the week". In this case the interviewer may have to tick more than one box for this question. This is known as multicoding or multiple responses.

Similarly, questions where the respondent’s opinion or motives are being asked (known as open-ended questions) do not have a list of possible answers (pre-coded questions) and therefore have to be coded very carefully.

It is important to have an understanding of the capabilities of the package chosen for analysis before starting to design the questionnaire.

6. Questionnaire design (and coding)

Questionnaire design and coding is the crucial part of the project and mistakes made here can cause serious problems at the analysis and reporting stages. It is a good idea to look at some examples of other questionnaires and how they are worded before starting to design a questionnaire. If you have ever been interviewed for a survey think about how you felt. Were the questions easy to answer or too complicated and were there enough options to choose from?

Here are a few points to consider.

• The questionnaire needs to be well set out and the spaces and boxes for entering the answers should be of a good size and positioned well.
• The questions should be phrased in a simple way that can easily be understood by the respondent.
• Keep questions short and avoid jargon.
• Read over the questions and check whether they can be misinterpreted. For example, the question "Have you visited the children’s play area recently?" doesn’t mention a specific time period so it is up to each respondent to interpret the word "recently". This could be a visit made in the last week, month or year.
• Open-ended questions (where the opinions or motives of the respondent are being sought) are difficult to answer and difficult to code and analyse. It is a good idea to limit the number of open-ended questions to about two or three per survey.
• To assess attitudes or views on a certain topic without using an open-ended question consider using a scale. An example of this would be, "How would you rate the cleanliness of the public toilets?" with the possible options being "very good/good/average/poor/very poor".
• Take care when wording personal or sensitive questions: it is better to say, "Please could you tell me your age on your last birthday?" and allow the respondent to point to an age band printed on the questionnaire rather than to ask them directly, "How old are you?"
• During a face-to-face interview it is possible to show the respondents pictures or maps if it will help them to answer the question more accurately. These are known as "showcards".

  Showcards can be a useful interviewing tool, but to keep the interview simple for the interviewer it is best to use them sparingly.

• Think about the flow of the questions. Put questions about similar topics together.
• Don’t make the questionnaire too long – a successful face-to-face interview lasts no longer than ten minutes.

7. Carrying out a pilot survey

Once the questionnaire is produced it is worth carrying out some test or "pilot" interviews to make sure that it is understandable, not too long, or produces answers not allowed for on the pre-coded questions.

The pilot could be carried out amongst colleagues at work, or if budgets allow, actually with visitors at the destination. After the pilot, it is important to assess how the questionnaire worked and make any necessary changes to wording or question order. It may be necessary to carry out another small pilot if the questionnaire has changed significantly.

8. Printing the questionnaire

Printing the questionnaire must only be done once the questionnaire has been finalised. Each interviewer will need a supply of questionnaires. Make sure that there is an adequate stock of these.

9. Training survey interviewers

Survey interviewers need to be capable of:

• approaching strangers and requesting their participation in the survey
• following the survey instructions
• recording information accurately.

For the best results it is preferable to use experienced interviewers. If using inexperienced staff it is important to make sure that they are briefed about the project and understand why the survey is being carried out and what is hoped to be learnt from it. It is also a good idea to make sure that everyone is clear about:

• where they will be interviewing (ie the location or locations in the destination)
• when and where to return their completed questionnaires
• who to call if they experience problems.

Most importantly, talk the interviewers through the questionnaire question by question and allow them to become familiar with it.

The organisation’s insurance must cover the interviewers for accident and public liability when they are working. It will also be necessary to set up a process to pay the interviewers for their work if they are not employees of the organisation or if the work is additional to their normal salary.

10. Collecting the data

Collecting the data should be one of the easiest stages in the whole survey process if the survey has been planned well. It is important that the interviewers are provided with the right equipment to do the job:

• clipboard
• supply of questionnaires
• maps and showcards (if you are using them)
• pencils, pencil sharpener, rubber
• identification letter or official badge (to prove that they are conducting an official survey on behalf of the organisation)
• waterproof covering for completed questionnaires and for use over the clipboard if it rains.

It is a good idea to have someone as a point of contact at all times during the interviewing period to deal with any problems or queries from the interviewers. It may be useful for that person to tour round the interview points to check on progress and monitor interview quality. Someone also needs to be available to collect completed questionnaires and store them safely at the end of each day.

11. Analysis and evaluation

A common source of error at the stage of analysis and evaluation can be inaccurate data entry. Typing in column after column of figures can be tedious and it is easy to lose concentration and skip a figure or column. It is therefore important to carry out a quality check on the data entry. This can be done by looking at about 10% of the questionnaires and checking their accuracy against the entered data. Obviously, if many errors are found the entire data set will need to be checked.

There are many agencies specialising in data entry and therefore sub-contracting this stage of the project may be possible if budget is available. Make use of all the functions in the chosen analysis package to help analyse the survey results, eg cross-tabulations, averages, percentages, charts.

12. The final report

After the data has been analysed it is good practice to write up the results of the survey in the form of a report. Before writing the report think again about who will be reading it and what the survey wanted to achieve. Include a copy of the questionnaire and any showcards used in the appendixes of the report.

13. Making sure issues are resolved

Don’t forget the reason for carrying out research in the first place is to monitor the views of visitors to a destination. If the results of the survey show that there is room for improvement in some of the services and facilities offered, then the next stage of the process is to begin to look at ways in which things can be changed. Celebrate the positive findings from the survey and examine how these practices can be implemented in other areas.

14. Monitoring changes made as a result of survey findings

Finally, if changes are implemented at the destination it is important to monitor how the visitors feel about them – and so the research process begins again.