Comprehensive exam topics
PhD Comprehensive Exam
Definitional Essay and Reading List
School of Communications
Janet Morrison, MA, MLIS
Mixed Methods Research Methodology
Over the past three decades, mixed methods research methodologies have enjoyed a steady increase in popularity as evidenced by the publication of numerous texts, book chapters and journal articles, support from grant funding agencies, and the launch of a peer reviewed journal (Journal of Mixed Methods Research). However, controversy still exists over the legitimacy of “mixing methods” and whether or not multi-methodological inquiry can rightly be called a paradigm in itself.
Central to the debate over the legitimacy of mixed or multi-methodological inquiry as a distinct research methodology is the famous (or perhaps infamous) discussion of the paradigmatic divide between quantitative and qualitative methodologies. “The divide” became a topic of debate in the 1980s and was synthesized by Lincoln and Guba in their influential book Naturalistic Inquiry (1985). Lincoln and Guba insisted that the two paradigms were so different in their ontologies, epistemologies and world views that mixing methods was an impossibility. This position became known as the ‘incompatibility thesis’ (Howe, 1988).
While the paradigm debate was largely resolved in the 1990’s it still flares up from time to time. Bergman (2008b) insists that the paradigm wars are not over – they have just reached a state of “détente” (p.2). Therefore, practitioners of mixed methods methodologies need to have a solid understanding of the ontological and epistemological foundations of mixed methods research in order to fully understand why and when the mixing of methods is appropriate and to defend their choice of research method in the face of the paradigm challenge. This comprehensive reading area will explore these philosophical underpinnings.
I propose to break the topic into six sections: first, an introduction, history and definition of mixed methods and mixed methodologies; second, a discussion of the paradigm issues which are commonly raised; third, an overview of pragmatism – the philosophical viewpoint which is most often identified with mixed methods research; fourth, a section on research design using mixed methodologies; fifth, a discussion of the critiques most often raised about ‘mixing methods’; and finally, concluding with a brief section on the future directions of mixed methods research.
1. Introduction, history and definition of mixed methods
To set the background for this discussion I have included the chapter by Teddlie and Johnson (2009) which provides a historical account of the evolution of research traditions and summarizes methodological thought of the 20th century. Second, the article by Johnson, Onwuegbuzie and Turner (2007) in the first volume of the Journal of Mixed Methods Research provides a comprehensive overview of the topic and includes a summary of definitions of mixed methods research by leaders in the field. I have also included the editorial contained in the premier issue of the Journal written by Tashakkori and Creswell (2007) that acknowledges the history of mixed methods. This topic is further expanded by the chapter by Teddlie and Tashakkori (2003). Finally Creswell and Plano Clark (2007c) provide an excellent time line which outlines the various periods of development of mixed methods research.
2. Paradigm issues and controversies - Dichotomy or continuum?
The debates around paradigm issues beg the question: should paradigms be viewed in a dichotomized fashion or as a continuum? In order to explore this debate I have included the following readings which explore the paradigm issue.
Lincoln & Guba (2000) provide a good foundational reading on the topic of methodological paradigms. Guba’s 1990 edited collection The Paradigm Dialog contained a series of essays which discussed the divide. I have included the first chapter The Alternative Paradigm Dialog since it provides a good summary of the philosophical underpinnings of positivism, post positivism, critical theory, and constructivism. Creswell (1998) Philosophical and theoretical frameworks also provides an excellent discussion of philosophical frameworks. I have also included Morgan, (2007) to provide a more balanced discussion of how paradigms shape world views, epistemology and shared beliefs.
Howe’s 1988 paper Against the qualitative-quantitative incompatibility thesis was a significant work in which he coined the term ‘the incompatibility thesis’ and refuted the incompatibility theory as dogma. Instead he makes a case for a pragmatic approach to research methodology a theme he expands upon in his later work Getting over the Quantitative – Qualitative Debate (1992).
Onwuegbuzie & Leech (2005) insist that there is considerable overlap between the two paradigms, and that perpetuating the notion of a ‘divide’ is counterproductive. Bergman (2008a) argues that research design is hampered by rigid thinking about the characteristics of the two paradigms. Many others including Brannen, 2005; Brewer & Hunter, 2006; Greene & Caracelli, 2003; Oakley, 1999; and Reichart & Rallis (1994) draw attention to the ways the paradigms are complimentary. As the final readings in this section Teddlie and Tashakorri’s (2003) Major issues and controversies in the use of mixed methods in the social sciences and their 2009 Paradigm Issues in Mixed Methods Research provide a good summary of the issues.
Pragmatism is the philosophical stance most often identified with mixed methods research. Maxcy (2003) traces the history of pragmatism and its contribution to social science research. The article by Johnson & Onwuegbuzie (2008) discusses the notion of pragmatism as the philosophical partner of mixed methods research explaining the general characteristics of pragmatism including its strengths and weaknesses as a justification for mixed methods research. This is echoed in the work of both Howe (1988) and Morgan (2007). In the final reading of this section Cherryholmes (1992, 1994) contrasts pragmatism with scientific realism and clarifies alternative views of pragmatism.
John Creswell has been a leader in mixed methods design and I have included two readings of his work (Creswell, 2003; Creswell & Plano Clark, 2007a). To conclude Tashakkori & Teddlie (2009) provide a comprehensive chapter covering the rationale for integrating methods.
As mentioned above, paradigm issues still arise in the critique of mixed methods research. One line of critique asserts that mixed methods research reflects positivism/post positivism (Sale, Lohfeld and Brazil, 2002) in fact Giddings (2006) insists that mixed methods research is “positivism in drag” (p.195) and that pragmatism is evoked merely as expediency in an attempt to curry favor with research funders.
Another line of critique points to methodological inconsistencies: the fact that mixed methods research studies are often poorly defined, fail to provide a rationale for combining methods, and in reality are merely two parallel studies conducted at the same time with little mixing of methods (Greene, 2008; Bryman, 2007) leading some authors to question the overall value and quality of mixed methods research (O’Cathain, Murphy & Nicholl, 2008).
6. Future directions
The criticism detailed above has sparked some authors to defend mixed methods pointing out that mixed methods design overcomes weakness in single or mono-method research and will facilitate research which is currently beyond current capabilities (Bergman, 2008b). Many other authors call for further refinements of methods to alleviate methodological inconsistencies discussed above (Greene, 2006, Yin, 2006). In conclusion to this section Creswell and Plano Clark (2007b) provide a comprehensive discussion of specific issues in mixed methods research that call for refinement and identify 8 topics for future research including sampling procedures, data analysis, software needs, and reporting of results
1. Bergman, M. (2008a). The straw men of the qualitative-quantitative divide and their influence on mixed methods research. In M. Bergman (Ed.), Advances in mixed methods research: Theories and applications (pp.11-21). London: Sage.
2. Bergman, M. (2008b). Whither mixed methods? In M. Bergman (Ed.), Advances in mixed methods research: Theories and applications (pp.1-7). London: Sage.
3. Brannen, J. (2005). Mixing methods: The entry of qualitative and quantitative approaches into the research process. International Journal of Social Research Methodology, 8(3), 173-184.
4. Brewer, J., & Hunter, A. (2006). A healthy skepticism about theory and method. In Foundations of multimethod research: Synthesizing styles (pp. 17-37). Thousand Oaks, CA: Sage.
5. Bryman, A. (2007). Barriers to integrating quantitative and qualitative research. Journal of Mixed Methods Research, 1(1), 8-22.
6. Cherryholmes, C. (1992). Notes on pragmatism and scientific realism. Educational Researcher, 21(6), 13-17.
7. Cherryholmes, C. (1994). More notes on pragmatism. Educational Researcher, 23(1), 16-18.
8. Creswell, J. (1998). Philosophical and theoretical frameworks. In J. Creswell Qualitative inquiry and research design: Choosing among five traditions (pp. 73-91). Thousand Oaks, CA: Sage.
9. Creswell, J. (2003). A framework for design. In J. Creswell Research design: Qualitative, quantitative and mixed methods approaches (2nd ed.). Thousand Oaks, CA: Sage.
10. Creswell, J., & Plano Clark, V. (2007a). Choosing a mixed method design. In J. Creswell & V. Plano Clark Designing and conducting mixed methods research (pp. 58-88). Thousand Oaks, CA: Sage.
11. Creswell, J., & Plano Clark, V. (2007b). Future directions for mixed methods research. In Designing and conducting mixed methods research (pp. 184-193). Thousand Oaks, CA: Sage.
12. Creswell, J., & Plano Clark, V. (2007c). Understanding mixed methods research. In J. Creswell & V. Plano Clark Designing and conducting mixed methods research (pp. 1-15). Thousand Oaks, CA: Sage.
13. Giddings, L. (2006). Mixed methods research: Positivism dressed in drag? Journal of Research in Nursing, 11(3), 195-205.
14. Greene, J. (2006). Toward a methodology of mixed methods social inquiry. Research in the Schools, 13(1), 93-98.
15. Greene, J. (2008). Is mixed methods social inquiry a distinctive methodology? Journal of Mixed Methods Research, 2(1), 7-22.
16. Greene, J., & Caracelli, V. (2003). Making paradigmatic sense of mixed methods practice. In A. Tashakkori & C. Teddlie Handbook of mixed methods in social & behavioral research (pp. 91-110). Thousand Oaks, CA: Sage.
17. Guba, E. (1990). The alternative paradigm dialog. In E. Guba (Ed. ). The paradigm dialogue. Newbury Park, CA: Sage.
18. Howe, K. (1988). Against the quantitative-qualitative incompatibility thesis or dogmas die hard. Educational Researcher, 17(8), 10-16.
19. Howe, K. (1992). Getting over the quantitative-qualitative debate. American Journal of Education, 100(2), 236-257.
20. Johnson, R., Onwuegbuzie, A., & Turner, L. (2007). Toward a definition of mixed methods research. Journal of Mixed Methods Research, 1, 112-133.
21. Johnson, R., & Onwuegbuzie, A. (2008). Mixed methods research: A research paradigm whose time has come. Educational Researcher, 33(7), 14-26.
22. Lincoln, Y., & Guba, E. (1985). Naturalistic Inquiry. Thousand Oaks, CA: Sage.
23. Lincoln, Y., & Guba, E. ( 2000). Paradigmatic controversies, contradictions, and emerging confluences. In N. Denzin & Y. Lincoln (Eds.), Handbook of qualitative research, (2nd ed., pp. 63-188). Thousand Oaks, CA: Sage.
24. Maxcy, S. (2003). Pragmatic threads in mixed methods research in the social sciences: The search for multiple modes on inquiry and the end of the philosophy of formalism. In A. Tashakkori & C. Teddlie Handbook of mixed methods in social & behavioral research (pp. 51-89). Thousand Oaks, CA: Sage.
25. Morgan, D. (2007). Paradigms lost and pragmatism regained: Methodological implications of combining qualitative and quantitative methods. Journal of Mixed Methods Research, 1(1), 48-76.
26. Oakley, A. (1999). Paradigm wars: Some thoughts on a personal and public trajectory. International Journal of Social Research Methodology, 2(3), 247-254.
27. O’Cathain, A., Murphy, E., & Nicholl, J. (2008). The quality of mixed methods studies in health services research. Journal of Health Services Research Policy, 13(2), 92-98.
28. Onwuegbuzie, A, & Leech, N. (2005) Taking the Q out of research: Teaching research methodology courses without the divide between quantitative and qualitative paradigms. Quality & Quantity, 39(3), 267-296.
29. Reichardt, C. & Rallis, S. (1994). Qualitative and quantitative inquiries are not incompatible: A call for a new partnership. New Directions for Program Evaluation, 61(1), 85-91.
30. Sale, J., Lohfeld, L., & Brazil, K. (2002). Revisiting the quantitative-qualitative debate: Implications for mixed methods research. Quality and Quantity, 36(1), 43-53.
31. Tashakkori, A., & Creswell, J. (2007). Editorial: The new era of mixed methods. Journal of Mixed Methods Research, 1(1), 3-7.
32. Tashakkori, A., & Teddlie, C. (2009). Integrating Qualitative and quantitative approaches to research. In L. Bickman & D. Rog (Eds.) Sage handbook of applied social research methods (pp. 283-317). Thousand Oaks, CA: Sage.
33. Teddlie, C., & Johnson, B. (2009). Methodological thought since the 20th century. In C. Teddlie & A. Tashakorri (Eds.), Foundations of mixed methods research: Integrating quantitative and qualitative approaches in the social and behavioral sciences (pp. 62-82). Thousand Oaks, CA: Sage.
34. Teddlie, C., & Tashakkori, A. (2003). Major issues and controversies in the use of mixed methods in the social and behavioral sciences. In A. Tashakkori & C. Teddlie Handbook of mixed methods in social & behavioral research (pp. 3-50). Thousand Oaks, CA: Sage.
35. Teddlie, C., & Tashakkori, A. (2009). Paradigm issues in mixed methods research. In C. Teddlie & A. Tashakkori Foundations of mixed methods research: Integrating Quantitative and qualitative approaches in the social and behavioral sciences. (pp.83-105). Thousand Oaks, CA: Sage.
36. Yin, R. ( 2006). Mixed methods research: Are the methods genuinely integrated or merely parallel? Education in the Schools 13(1), 41-47.