As a researcher, you will be investigating the unknown. Undertaking that investigation systematically and meticulously is a significant part of research and may involve using complex methods to ensure that those methods will produce the results you need. Piloting your chosen method will help you to identify any potential flaws and afford you the opportunity to redress them. In this chapter we will show you how to pilot your selected method and test its efficiency. Engaging with this chapter will enable you to develop an effective pilot study and critically assess the outcomes of that pilot. The companion website will present you with the experiences of researchers who have used pilot studies.
Hi, my name is Dr Gareth Davies, I’m from Lews Castle College UHI, and I’m based in Stornoway. My primary research area is in health psychology and that involved qualitative and quantitative work. I’m going to talk to you today about pilot studies, and how you can use pilot studies in your research, and how I use pilot studies in my research.
It’s important to say that pilot studies involve an investment in time. However, the small investment of time is almost certain to pay off in the long run. If you do a pilot study, and you don’t find any problems or issues with the data-gathering that you’ve just undertaken, the analysis of that data that you’ve just gathered, then there’s no reason why you can’t include that data in a bigger study.
However, what we’re going to do is focus on the pilot study itself. The purpose of the pilot study is to test your data gathering and analysis strategies. Your research questions should really inform your data gathering and analysis strategy. It’s important to understand that you should think about your data analysis long before you collect any data. The pilot study offers you a great opportunity to test those data analysis strategies that you’ve decided on, before you actually go ahead and do your main study.
I’m going to talk about pilot studies in quantitative research and then I’m going to talk about pilot studies in qualitative research. To begin with, in quantitative research, it enables you to ensure that you’ve got the correct data gathering systems in place. For instance, if you’re using a questionnaire, it enables you to test whether a questionnaire will actually do what you want it to do. It will enable you to test whether the questions you’ve got in that questionnaire are actually fit for purpose, or whether they are indeed superfluous to requirements. When you’re actually doing some inferential statistics with some pilot study data, it’s not uncommon to use a simpler or more forgiving significance level. It’s industry standard to use the 0.05 significance level in disciplines such as psychology for instance. In a pilot study you can actually use an easier significance level. This enables you to see whether you’re actually identifying any trends in data, or something along those lines. It just gives you a little more flexibility as well. Of course, if you do decide that the data that you’ve gathered is what you want, then you can go back and reanalyse that, using the more stringent 0.05 significance level.
Obviously, the pilot study will help you identify any problems that you’ve got in your procedure and enable you to correct that before you actually gather any more data in the long run.
So qualitative pilot studies then – qualitative studies, as you know are going to be different from quantitative ones, in that they are based around interviews etc. In qualitative studies, you might for instance be using a semi structured interview approach, and that might be your data gathering strategy. So when you would do your interview, you’d be using an interview schedule, and the pilot study enables you to test whether that interview schedule does what you want it to do. In other words, whether it helps you gather data that you need to answer your research question. A qualitative pilot study would be useful if you were to for instance, go out and ask a colleague or a critical friend if they wouldn’t mind being the participant in your pilot study interview. This enables you to ask all the questions in a nice relaxed atmosphere and it also enables you to identify if any of the questions that you’ve come up with are not really going to do what you want, whether they might be answered far too simplistically, or whether they are far too complex for instance.
When you analyse qualitative data, the industry standard way of qualitative data is to use Richie and Spencer’s framework analysis. Now, if you’ve got a test piece of interview you can transcribe that, then you can work through that using your framework analysis, and then you can figure out if you need to make any changes to your interview schedule, or whether in fact the questions that you’ve asked have elicited the type of data that you want. Of course, both qualitative and quantitative pilot studies, they enable you to bring out any issues or problems before you go ahead and do your main study. If your pilot study does reveal some issues and problems, if they’re not too significant then you might just want to make some minor alterations. If indeed it reveals something a little bit more fundamental as a problem in your data gathering analysis strategy, you are then able to go back and perform another pilot study to ensure that you have ironed out those problems. I hope that helps you with pilot studies, and good luck with your own subsequent research.
The University of Surry have produced this PDF which is a couple of pages long. It examines pilot studies; what they are, why they are important and what can be achieved by conducting them. There are bundles of useful references in the comprehensive reference section.
A link to an extensive set of notes from the University of Surrey dealing with the whole process of documenting research. It starts with a consideration of why we do research, works through a range of planning issues that need to be considered, and concludes with a section on how research result might be utilised. Contains some embedded links.
The contents are well-defined and easy to read, in short, informative sub-sections. Not all of the sections will be appropriate to every research project, so students will need to use their own judegement, but taken as a whole, the document gives a good overview of the kinds of issues which students need to consider when preparing a research dissertation.
These are helpfully broken-up into separate tasks, such as preparing a proposal, writing a report, editing, and so on. Each video clip gives a general overview and some guidance for students to help manage the selected task.
Individual videos deal with different forms of research, evaluation, and project planning. The clips range from 30 minutes to an hour each, so you may wish to watch them in stages, or allow time to watch, pause, and take notes. Different videos will be appropriate to different subject matter, so you may want to discuss your initial options with your supervisor before you invest time learning a research method which is not suitable for the topic of your study.
