As we look ahead to ICQI 2017, we are sharing a few more reflections on this year’s conference. Here, Christina Silver and Nicholas Woolf describe their work on Five-Level QDA.
In the Digital Tools Stream at ICQI 2016 we presented two papers discussing our work to develop and implement the Five-Level QDASM Method, a CAQDAS pedagogy that transcends methodologies, software programs and teaching modes. The first, called “Operationalizing our responsibilities: equipping universities to embed CAQDAS into curricular” was presented by Christina in the opening plenary session. The second, called “Five-level QDA: A pedagogy for improving analysis quality when using CAQDAS” was presented jointly by Nicholas and Christina. Here we briefly summarize these two papers. You can find out more about the Five-Level QDA method and our current work by visiting our website.
Responsibilities for effective CAQDAS teaching in the digital age
There is an expanding range of digital tools to support the entire process of undertaking qualitative and mixed methods research, and current generations of students expect to use them (Paulus et al., 2014), whatever their disciplinary, methodological or analytic context. Although many researchers use general-purpose programs to accomplish some or all of their analysis, we focus on dedicated Computer Assisted Qualitative Data AnalysiS (CAQDAS) packages. CAQDAS packages are now widely used and research illustrates that uptake continues to increase (White et al. 2012; Gibbs, 2014; Woods et al. 2015). However, there’s little evidence that their use is widely embedded into university curricula. There may be several reasons for this (Gibbs, 2014), including the difficulty of attending to diverse learner needs, which are affected by learners’ methodological awareness, analytic adeptness and technological proficiency (Silver & Rivers, 2015).
These issues highlight the importance of developing effective ways of embedding CAQDAS teaching into university curricula. This long-standing issue has been debated for as long as these programs have been available, and it is widely agreed that the appropriate use of digital technologies must be taught in the context of methodology (Davidson & Jacobs, 2008; Johnston, 2006; Kuckartz, 2012; Richards, 2002; Richards & Richards, 1994; Silver & Rivers, 2015; Silver & Woolf, 2015). However, few published journal articles illustrate teaching of CAQDAS packages as part of an integrated methods courses (recent exceptions are Paulus & Bennett, 2015; Bourque & Bourdon, 2016 and Leitch et al., 2016). Several reflective discussions regarding the integration of methodological, analytical and technological teaching are insightful and useful (e.g., Carvajal, 2002; Walsh, 2003; Davidson & Jacobs, 2008; di Gregorio & Davidson, 2008), as are concrete examples of course content, modes of delivery, course assignments and evaluation methods (e.g., Este et al., 1998; Blank, 2004; Kaczynski & Kelly, 2004; Davidson et al., 2008; Onwuegbuzie et al., 2012; Leitch et al., 2015; Paulus & Bennett, 2015; Bourque & Bourdon, 2016; Jackson, 2003). However, these writers provide varying degrees of detail about instructional design and are contextually specific, focusing on the use of a particular CAQDAS program, a disciplinary domain, and/or a particular analytic framework. Their transferability and pedagogical value may therefore be limited where there is an intention to use different methodologies, analytic techniques and software programs.
There are clearly challenges and lack of guidance in the literature for teaching qualitative methodology, analytic technique and technology concurrently. Although the challenges are real they need not be seen as barriers. A pedagogy that transcends methodologies, analytic techniques, software packages and teaching modes could prompt a step-change in the way qualitative research in the digital environment is taught (Silver & Woolf, 2015). The Five-Level QDA method is designed as such a pedagogy with the explicit goal of addressing these challenges.
The Five-Level QDA Method: a CAQDAS pedagogy that spans methodologies, software packages and teaching modes
The Five-Level QDA method (Woolf, 2014; Silver & Woolf, 2015; Woolf & Silver, in press) is a pedagogy for learning to harness CAQDAS packages powerfully. The phrase “harness CAQDAS packages powerfully” isn’t just a fancy way of saying “use CAQDAS packages well”, but means using the chosen software from the start to the end of a project, while remaining true throughout to the iterative and emergent spirit of qualitative and mixed methods research. It isn’t a new or different way of undertaking analysis, but explicates the unconscious practices of expert CAQDAS users, developed from our experience of using, teaching, observing and researching these software programs over the past two decades. It involves a different way of harnessing computer software from a taken-for-granted or common sense approach of simply observing the features on a computer screen and looking for ways of using them.
The principles behind the Five-Level QDA Method
The core principle is the need to distinguish analytic strategies – what we plan to do – from software tactics – how we plan to do it. As uncontroversial as this sounds, strategies and tactics in everyday language are commonly treated as synonyms or near-synonyms, leading unconsciously to the methods of a QDA and the use of the CAQDAS package’s features being considered together as a single process of what we plan to do and how we plan to do it. A consequence of this is that the features of the software to either a small or a large degree drive the analytic process.
The next principle is recognizing the contradiction between the nature of CAQDAS which is to varying degrees iterative and emergent, and the predetermined steps or cut-and-dried nature of computer software. When this is not consciously recognized, either the strategy is privileged, with the consequence that the software is not used to its full potential throughout a project; or the tactics are privileged, with the consequence that the iterative and emergent aspects of a QDA are suppressed to some degree. However, when the contradiction is consciously recognized it becomes necessary to reconcile it in some way. One approach is through a compromise, or trade-off, in which the analytic tasks of a project are raised to a more general level and expressed as a generic model of data analysis in order to more easily match the observed features of CAQDAS packages (terms in italics have a specific meaning in Five-Level QDA).
The Five-Level QDA method, following Luttwak’s (2001) five level model of military strategy, takes a different approach to reconciling the contradiction between strategies and tactics by placing it in a larger context in order to transcend the contradiction. Regardless of research design and methodology, there are two levels of strategy – the project’s methodology and objectives (Level 1), and the analytic plan (Level 2) that arises from those objectives. There are similarly two levels of tactics – the straightforward use of software tools (Level 4) and the sophisticated use of tools (Level 5). We use the term tools in a particular way. We are not referring to software features, but ways of acting on software components – things in the software that can be acted upon. Whereas CAQDAS packages have hundreds of features, they have far fewer components, typically around 15-20.
The critical middle level between the strategies and tactics is the process of translation (Level 3).Rather than raise the level of analytic tasks to the level of software features, the level of analytic tasks is lowered to the level of its units, which are then matched, or translated, to the components of the CAQDAS package. This method ensures that the direction of action of the process is always initiated in a single direction: from analytic strategies to software tactics, and never the other way around. This ensures that the analytic strategies drive the analytic process, not the available features of the software. Because translation operates at the level of individual analytic tasks the method is relevant across methodologies and software programs. Figure 1 provides an overview of the Five-Level QDA method.
Figure 1. Five-Level QDA Chart
|two levels of strategy >>>>> translated to >>>>> two levels of tactics|
|Level 1||Level 2||Level 3||Level 4||Level 5|
The purpose and context of a project, usually expressed as research questions and a methodology
The conceptual framework and resulting analytic tasks
Translating from analytic tasks to software tools, and translating the results back again
Straightforward choice of individual software operations
Sophisticated use of software by combining operations or performing them in a custom way
Several people at our presentations fed-back to us that this chart implies the process is linear and hierarchical, which of course is misleading because all forms of QDA are to varying degrees iterative and emergent. Since ICQI, therefore, we have created a diagram that we hope more accurately reflects the iterative, cyclical nature of the process (Figure 2).
Figure 2. Five-Level QDA diagram
Tools for teaching the Five-Level QDA Method
In order to illustrate translation in the context of learners’ own projects we have developed two displays: Analytic Overviews (AOs) and Analytic Planning Worksheets (APWs). AOs provide a framework for the development of a whole project, described at strategies Levels 1 and 2, whereas APWs scaffold the process of translation, facilitating the skill of matching the units of analytic tasks to software components in order to work out whether a software tool can be selected, or needs to be constructed. We don’t have space here to illustrate translation in detail, but further details about teaching the Five-Level QDA method via the use of AOs and APWs are discussed in Silver & Woolf (2015) and Woolf & Silver (in press).
Implementing and researching the Five-Level QDASM Method
The Five-Level QDA method provides an adaptable method for teaching and learning CAQDAS. Since 2013 we’ve been using it in our own research and teaching in many different contexts, and we’ve used feedback from our students and peers to refine the AOs and APWs. We’re also and working with several universities to improve provision of CAQDAS teaching, using the Five-Level QDA method in different learning contexts.
We believe that the Five-Level QDA method is adaptable to a range of instructional designs including different face-to-face workshop designs and remote learning and via textbook and complimentary video-tutorials (Woolf & Silver, in press). This is because it provides a framework through which qualitative and mixed methods research and analysis can be taught at the strategies level within the context of digital environments. Instructional designs that teach qualitative methodology and analytic techniques via the use of CAQDAS packages illustrate that qualitative methodology and technology need not be introduced to students separately (e.g. Davidson et al, 2008, Bourque 2016, Leitch 2016). The Five-Level QDA method pre-supposes that it is not possible to adequately teach technology without methodology, but we would also argue that in our increasingly digital environment it is increasingly less acceptable to adequately teach methodology without technology. The intentionally separate emphasis given to analytic strategies and software tactics within a single framework enables the teaching of methodology and technology concurrently within an instructional design that is adaptable to local contexts, as well as serving as a method to harness CAQDAS for researchers’ own projects.
We are currently evaluating our work and welcome opportunities to work with others to implement and further research the effectiveness of the Five-Level QDA method in different contexts.
Blank, G. (2004). Teaching Qualitative Data Analysis to Graduate Students. Social Science Computer Review, 22(2), 187–196. http://doi.org/10.1177/0894439303262559
Bourque, C. J., & Bourdon, S. (2016). Multidisciplinary graduate training in social research methodology and computer-assisted qualitative data analysis: a hands-on/hands-off course design. Journal of Further and Higher Education, 9486(April), 1–17. http://doi.org/10.1080/0309877X.2015.1135882
Carvajal, D. (2002). The Artisan ’ s Tools . Critical Issues When Teaching and Learning CAQDAS. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 3(2, Art 14).
Davidson, J. (n.d.). Thinking as a Teacher : Fully Integrating NVivo into a Qualitative Research Class.
Davidson, J., & Jacobs, C. (2008). The Implications of Qualitative Research Software for Doctoral Work. Qualitative Research Journal, 8(2), 72–80. http://doi.org/10.3316/QRJ0802072
Davidson, J., Jacobs, C., Siccama, C., Donohoe, K., Hardy Gallagher, S., & Robertson, S. (2008). Teaching Qualitative Data Analysis Software ( QDAS ) in a Virtual Environment : Team Curriculum Development of an NVivo Training Workshop. In Fourth International Congress on Qualitative Inquiry (pp. 1–14).
Este, D., Sieppert, J., & Barsky, A. (1998). Teaching and Learning Qualitative Research With and Without Qualitative Data Analysis Software. Journal of Research on Computing in Education, 31(2), 17. http://doi.org/10.1080/08886504.1998.10782247
Gibbs, G. R. (2014). Count: Developing STEM skills in qualitative research methods teaching and learning. Retrieved from https://www.heacademy.ac.uk/sites/default/files/resources/Huddersfield_Final.pdf
Jackson, K. (2003). Blending technology and methodology. A shift towards creative instruction of qualitative methods with NVivo. Qualitative Research Journal, 3(Special Issue), 15.
Johnston, L. (2006). Software and Method: Reflections on Teaching and Using QSR NVivo in Doctoral Research. International Journal of Social Research Methodology, 9(5), 379–391. http://doi.org/10.1080/13645570600659433
Kaczynski, D., & Kelly, M. (2004). Curriculum Development for Teaching Qualitative Data Analysis ONline. QualIT 2004: International Conference on Qualitative Research in IT and IT in Qualitative Research, (November), 9.
- Leitch, J., Oktay, J., & Meehan, B. (2015). A dual instructional model for computer-assisted qualitative data analysis software integrating faculty member and specialized instructor: Implementation, reflections, and recommendations. Qualitative Social Work. http://doi.org/10.1177/1473325015618773
Onwuegbuzie, A. J., Leech, N. L., Slate, J. R., Stark, M., Sharma, B., Frels, R., … Combs, J. P. (2012). An exemplar for teaching and learning qualitative research. The Qualitative Rport, 17(1), 646–647. Retrieved from http://www.nova.edu/ssss/QR/QR17-1/onwuegbuzie.pdf
Paulus, T. M., & Bennett, A. M. (2015). “I have a love–hate relationship with ATLAS.ti”TM: integrating qualitative data analysis software into a graduate research methods course. International Journal of Research & Method in Education, (June), 1–17. http://doi.org/10.1080/1743727X.2015.1056137
Silver, C., & Rivers, C. (2015). The CAQDAS Postgraduate Learning Model: an interplay between methodological awareness, analytic adeptness and technological proficiency. International Journal of Social Research Methodology, 5579(September), 1–17. http://doi.org/10.1080/13645579.2015.1061816
Silver, C., & Woolf, N. H. (2015). From guided-instruction to facilitation of learning : the development of Five-level QDA as a CAQDAS pedagogy that explicates the practices of expert users. International Journal of Social Research Methodology, (July 2015), 1–17. http://doi.org/10.1080/13645579.2015.1062626
Walsh, M. (2003). Teaching Qualitative Analysis Using QSR NVivo 1. The Qualitative Report, 8(2), 251–256. Retrieved from http://www.nova.edu/ssss/QR/QR8-2/walsh.pdf
White, M. J., Judd, M. D., & Poliandri, S. (2012). Illumination with a Dim Bulb? What do social scientists learn by employing qualitative data analysis software (QDAS) in the service of multi-method designs? Sociological Methodology, 42(1), 43.–76. http://doi.org/10.1177/0081175012461233
Woods, M., Paulus, T., Atkins, D. P., & Macklin, R. (2015). Advancing Qualitative Research Using Qualitative Data Analysis Software (QDAS)? Reviewing Potential Versus Practice in Published Studies using ATLAS.ti and NVivo, 1994–2013. Social Science Computer Review, 0894439315596311. http://doi.org/10.1177/0894439315596311
Woolf, N. H., & Silver, C. (in press). Qualitative analysis using ATLAS.ti: The Five-Level QDA Method. London: Routledge.
Woolf, N. H., & Silver, C. (in press). Qualitative analysis using MAXQDA: The Five-Level QDA Method. London: Routledge.
Woolf, N. H., & Silver, C. (in press). Qualitative analysis using NVivo: The Five-Level QDA Method. London: Routledge.