(SHI98). 0000011904 00000 n
Indicative of an apparent oscillation of qualitative researchers between the different requirements and protocols demarcating the quantitative and qualitative worlds, there were a few instances of articles which briefly recognised their small sample size as a limitation, but then defended their study on more qualitative grounds, such as their ability and success at capturing the complexity of experience and delving into the idiographic, and at generating particularly rich data. Glaser BG, Strauss AL. Ritchie J, Lewis J, Elam G. Designing and selecting samples. An examination of 83 qualitative interview studies in leading information systems journals [30] indicated little defence of sample sizes on the basis of recommendations by qualitative methodologists, prior relevant work, or the criterion of saturation. Critiquing the concept of saturation, Nelson [19] proposes five conceptual depth criteria in grounded theory projects to assess the robustness of the developing theory: (a) theoretical concepts should be supported by a wide range of evidence drawn from the data; (b) be demonstrably part of a network of inter-connected concepts; (c) demonstrate subtlety; (d) resonate with existing literature; and (e) can be successfully submitted to tests of external validity. group interviews, structured interviews and e-mail interviews over a period of time were excluded), and the data had to be analysed qualitatively (i.e. 2014;11(5):51325. Rather, sample size seemed to correlate with factors such as the journal of publication or the region of study (US vs Europe vs Asia). Despite this, although nuances within sub-themes were still emerging towards the end of data analysis, the themes themselves were being replicated indicating a level of completeness. That ones sample might be thought small was often construed as a limitation couched in a discourse of regret or apology. respondents expertise it should be acknowledged that this study was based on a small sample of self-selected couples in enduring marriages who were not broadly representative of the population. 2017;17(5):57188. The present research, therefore, incorporates a comparative element on the basis of three different disciplines engaging with qualitative health research: medicine, psychology, and sociology. 2015;25(5):5878. National Centre for Research Methods Review Paper. Descriptions of sampling practices within five approaches to qualitative research in education and the health sciences. 0000001969 00000 n
Finally, SHI112 (see extract in section Further sampling to check findings consistency) argued that it achieved saturation of discursive patterns. Another perspective on guest, bunce, and Johnsons (2006) landmark study. respondents distribution aleksandra Francis JJ, Johnston M, Robertson C, Glidewell L, Entwistle V, Eccles MP, et al. 0000010124 00000 n
0000007411 00000 n
2003;15(4):386404. Borsci S, Macredie RD, Martin JL, Young T. How many testers are needed to assure the usability of medical devices? 0000004729 00000 n
respondents gender 2010. Thousand Oaks, CA: Sage; 1998. 1996;19(6):5259. Although a small-scale qualitative study related to school counselling, this analysis can be usefully regarded as a case study of the successful utilisation of mental health-related resources by adolescents. Table1 provides basic information about the sample sizes measured in number of interviews of the studies reviewed across the three journals. respondents sdg isokan attainments perceived
2015;18(6):6856. Quality and trustworthiness in qualitative research in counseling psychology. For example, SHI139, in line with the justification of thematic saturation that it offered, expressed trust in its sample size sufficiency despite the poor response rate. (BMJ13). We selected journals which (a) follow a peer review process, (b) are considered high quality and influential in their field as reflected in journal metrics, and (c) are receptive to, and publish, qualitative research (AdditionalFile1 presents the journals editorial positions in relation to qualitative research and sample considerations where available). On the basis of the researchers previous experience and the literature, [30, 31] we estimated that recruitment of 1520 patients at each site would achieve data saturation when data from each site were analysed separately. Only, 12% of mentions of sample size justification pertained to arguments that were external to the study at hand, in the form of existing sample size guidelines and prior research that sets precedents. Editorial positions on qualitative research and sample considerations (where available). Konstantina Vasileiou. This research was initially conceived of and partly conducted with financial support from the Multidisciplinary Assessment of Technology Centre for Healthcare (MATCH) programme (EP/F063822/1 and EP/G012393/1). (SHI32).
Those who characterised their sample as small connected this to the limited potential for generalization of the results. ); (b) information about the aims of the study, the sample size and any justification for this, the participant characteristics, the sampling technique and any sample-related observations or comments made by the authors; and (c) information about the method or technique(s) of data analysis, the number of researchers involved in the analysis, the potential use of software, and any discussion around epistemological considerations. These data may be seen as an opportunity to generate further hypotheses and are a unique application of the medicalisation framework. 0000013041 00000 n Within journals, Mann-Whitney tests indicated that sample sizes of justifying and non-justifying articles in the BMJ and SHI did not differ significantly from each other.
(SHI139). These narratives were evident both in papers that justified their sample size and those that did not. 0000014581 00000 n
Qual Res. 2017;29(1):2341. 2001;24(3):23040.
California Privacy Statement, Moreover, a high degree of consensus had begun to emerge among those interviewed, and while it is always difficult to judge at what point theoretical saturation has been reached, or how many interviews would be required to uncover exception(s), it was felt the number was sufficient to satisfy the aims of this small in-depth investigation (Strauss and Corbin 1990). ACM Trans Comput Hum Interact. questionnaire respondents 1995;311(6999):2513. ACM. qualities of the analysis, meeting sampling or research design requirements, richness and volume of the data obtained, nature of study, further sampling to check findings consistency). respondents proportions preferring 1995;17(1):89113. a 15-year review period). Finally, one BMJ paper (BMJ21) defended its sample size with reference to the complexity of the analytic task. Despite these limitations, the analysis of the characterisation of sample size and of the threats seen to accrue from insufficient sample size, enriches our understanding of sample size (in)sufficiency argumentation by linking it to other features of the research. respondents matrilineal quantitative reproductive Although IPA generally involves intense scrutiny of a small number of transcripts, it was decided to recruit a larger diverse sample as this is the first qualitative study of this population in the United Kingdom (as far as we know) and we wanted to gain an overview. : a review of qualitative interviews in IS research. These thirty, from which we draw our findings, were chosen for translation based on heterogeneity in depressive symptomology and educational attainment. Mixed method studies and articles reporting more than one qualitative method of data collection (e.g. Finally, the pre-determination of sample size on the basis of sampling requirements was stated by one article though this was not used to justify the number of interviews (SHI10). respondents Article (DOCX 12 kb), List of eligible articles included in the review (N=214). 2006;18(1):5982. Sandelowski M. Real qualitative researchers do not count: the use of numbers in qualitative research. Sampling for qualitative research. (SHI57). The size of the sample was largely determined by the availability of respondents and resources to complete the study. (SHI131). qualitative interviews entrepreneurial quantitative respondents competencies D#HtTZH+X$u & Constantinou CS, Georgiou M, Perdikogianni M. A comparative method for themes saturation (CoMeTS) in qualitative interviews. The determination of sample size on the basis of pragmatic considerations was the second most frequently invoked argument (9.6% of all justifications) appearing in all three journals. Methodological studies in this area aim to provide guidance about saturation and develop a practical application of processes that operationalise and evidence saturation. Ericksons [62] dimensions of evidentiary adequacy are useful here. Google Scholar. We recommend, firstly, that qualitative health researchers be more transparent about evaluations of their sample size sufficiency, situating these within broader and more encompassing assessments of data adequacy. Characterising and justifying sample size sufficiency in interview-based studies: systematic analysis of qualitative health research over a 15-year period. Soc Sci Med. Qualitative health research is an inter-disciplinary field that due to its affiliation with medical sciences, often faces views and positions reflective of a quantitative ethos. Qual Health Res. A further systematic analysis [32] of health education research over 20years demonstrated that interview-based studies averaged 104 participants (range 2 to 720 interviewees). In: Ritchie J, Lewis J, editors. Sample sizes of BMJ and SHI articles did not differ significantly from each other. Choosing a suitable sample size in qualitative research is an area of conceptual debate and practical uncertainty. 2030 interviews) and single case (i.e. Using this approach it was possible to more carefully explore childrens discourse about the I, agency, relationality and power in the thematic areas, revealing the subtle discursive variations described in this article. Chicago, IL: Aldine; 1967. As the peer-review process becomes increasingly public, future research could usefully examine how reporting around sample size sufficiency and data adequacy might be influenced by the interactions between authors and reviewers. Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. Recommendations for designing and reviewing qualitative research in psychology: promoting methodological integrity. Thus, participants may not be representative of couples that experience postnatal PTSD. J Comput Inform Syst. (BJHP31). Accessed 17 May 2018. In a theory-driven content analysis, Francis et al. That sample size principles, guidelines and tools have been developed to enable researchers to set, and justify the acceptability of, their sample size is an indication that the issue constitutes an important marker of the quality of qualitative research. The discovery of grounded theory: strategies for qualitative research. One paper claimed theoretical saturation (BMJ06) conceived as being when no further recurring themes emerging from the analysis whilst another study argued that although the analytic categories were highly saturated, it was not possible to determine whether theoretical saturation had been achieved (BMJ04). quantitative easing We note, first, the limited number of health-related journals reviewed, so that only a snapshot of qualitative health research has been captured. The past decade has seen a growing appetite in qualitative research for an evidence-based approach to sample size determination and to evaluations of the sufficiency of sample size. BMJ. Qual Health Res. A minority of articles discussed how sample-related limitations (with the latter most often concerning the type of sample, rather than the size) limited generalizability. Additional File 2 lists all eligible studies that were included in the present analysis. This study focused on British Chinese carers of patients with affective disorders, using a qualitative methodology to synthesise the sociocultural representations of illness within this community. 2005;15(9):127788. 2014;11(1):2541. 2014;120:13541. Bowen GA. Naturalistic inquiry and the saturation concept: a research note. (SHI23). PLoS Med 2009; 6(7): e1000097. interview studies nested in a trial, as part of a broader ethnography, as part of a longitudinal research) were included if they reported only single-time qualitative interviews. 0000001056 00000 n Nevertheless, this more formulaic approach raised criticisms relating to assumptions about the conceptual [43] and ontological status of themes [44] and the linearity ascribed to the processes of sampling, data collection and data analysis [45]. 3) that needs to be further conducted, the analysis of which will not yield any new themes or ideas. Sandelowski M. One is the liveliest number: the case orientation of qualitative research. Four BJHP studies, all adopting IPA, expressed the appropriateness or desirability of small sample sizes (BJHP41; BJHP45) or hastened to explain why they included a larger than typical sample size (BJHP32; BJHP47). 2000;10(1):35. Others investigated PhD qualitative studies [31] and grounded theory studies [35]. We set a target of seven to 10 caregivers per site because of time constraints and the anticipated difficulty of accessing caregivers at some home based care services. Morse JM. The Abstract, Methods and Discussion (and/or Conclusion) sections of each article were examined by one author (KV) who extracted all the relevant information. 2016;18(1):47. The current study employed a sample of 10 in keeping with the aim of exploring each participants account (Smith et al., 1999). Given that older adults have been ignored in the research concerning suntanning, fair-skinned older adults are the most likely to experience skin cancer, and women privilege appearance over health when it comes to sunbathing practices, our study offers depth and richness of data in a demographic group much in need of research attention. In the BJHP, justifying articles (Mean rank=31.3) had significantly larger sample sizes than non-justifying studies (Mean rank=22.7; U=237.000, p<.05). Field Methods. Secondly, we invite researchers critically to consider how saturation parameters found in prior methodological studies and sample size community norms might best inform, and apply to, their own project and encourage that data adequacy is best appraised with reference to features that are intrinsic to the study at hand. (SHI127). (BJHP41). In the BMJ, one article (BMJ15) appealed to pragmatic reasons, relating to time constraints and the difficulty to access certain study populations, to justify the determination of its sample size. Despite increasing attention to rigor in qualitative research (e.g. 2017;27(4):591608. Finally, three SHI articles explained their sample size with reference to practical aspects: time constraints and project manageability (SHI56), limited availability of respondents and project resources (SHI131), and time constraints (SHI113). In total, 214 articles 21 in the BMJ, 53 in the BJHP and 140 in the SHI were eligible for inclusion in the review. In: Wittrock M, editor. Pragmatic considerations, such as resource constraints or participant response rate and availability, was the second most frequently used argument accounting for approximately 10% of justifications and another 23% of justifications also represented intrinsic-to-the-study characteristics (i.e. households satisfaction Are we there yet? Only one article (SHI30) used the term transferability to argue for the potential of wider relevance of the results which was thought to be more the product of the composition of the sample (i.e.
Finally, SHI112 (see extract in section Further sampling to check findings consistency) argued that it achieved saturation of discursive patterns. Another perspective on guest, bunce, and Johnsons (2006) landmark study. respondents distribution aleksandra Francis JJ, Johnston M, Robertson C, Glidewell L, Entwistle V, Eccles MP, et al. 0000010124 00000 n
0000007411 00000 n
2003;15(4):386404. Borsci S, Macredie RD, Martin JL, Young T. How many testers are needed to assure the usability of medical devices? 0000004729 00000 n
respondents gender 2010. Thousand Oaks, CA: Sage; 1998. 1996;19(6):5259. Although a small-scale qualitative study related to school counselling, this analysis can be usefully regarded as a case study of the successful utilisation of mental health-related resources by adolescents. Table1 provides basic information about the sample sizes measured in number of interviews of the studies reviewed across the three journals. respondents sdg isokan attainments perceived 
2015;18(6):6856. Quality and trustworthiness in qualitative research in counseling psychology. For example, SHI139, in line with the justification of thematic saturation that it offered, expressed trust in its sample size sufficiency despite the poor response rate. (BMJ13). We selected journals which (a) follow a peer review process, (b) are considered high quality and influential in their field as reflected in journal metrics, and (c) are receptive to, and publish, qualitative research (AdditionalFile1 presents the journals editorial positions in relation to qualitative research and sample considerations where available). On the basis of the researchers previous experience and the literature, [30, 31] we estimated that recruitment of 1520 patients at each site would achieve data saturation when data from each site were analysed separately. Only, 12% of mentions of sample size justification pertained to arguments that were external to the study at hand, in the form of existing sample size guidelines and prior research that sets precedents. Editorial positions on qualitative research and sample considerations (where available). Konstantina Vasileiou. This research was initially conceived of and partly conducted with financial support from the Multidisciplinary Assessment of Technology Centre for Healthcare (MATCH) programme (EP/F063822/1 and EP/G012393/1). (SHI32).
Those who characterised their sample as small connected this to the limited potential for generalization of the results. ); (b) information about the aims of the study, the sample size and any justification for this, the participant characteristics, the sampling technique and any sample-related observations or comments made by the authors; and (c) information about the method or technique(s) of data analysis, the number of researchers involved in the analysis, the potential use of software, and any discussion around epistemological considerations. These data may be seen as an opportunity to generate further hypotheses and are a unique application of the medicalisation framework. 0000013041 00000 n Within journals, Mann-Whitney tests indicated that sample sizes of justifying and non-justifying articles in the BMJ and SHI did not differ significantly from each other.
(SHI139). These narratives were evident both in papers that justified their sample size and those that did not. 0000014581 00000 n
Qual Res. 2017;29(1):2341. 2001;24(3):23040.
California Privacy Statement, Moreover, a high degree of consensus had begun to emerge among those interviewed, and while it is always difficult to judge at what point theoretical saturation has been reached, or how many interviews would be required to uncover exception(s), it was felt the number was sufficient to satisfy the aims of this small in-depth investigation (Strauss and Corbin 1990). ACM Trans Comput Hum Interact. questionnaire respondents 1995;311(6999):2513. ACM. qualities of the analysis, meeting sampling or research design requirements, richness and volume of the data obtained, nature of study, further sampling to check findings consistency). respondents proportions preferring 1995;17(1):89113. a 15-year review period). Finally, one BMJ paper (BMJ21) defended its sample size with reference to the complexity of the analytic task. Despite these limitations, the analysis of the characterisation of sample size and of the threats seen to accrue from insufficient sample size, enriches our understanding of sample size (in)sufficiency argumentation by linking it to other features of the research. respondents matrilineal quantitative reproductive Although IPA generally involves intense scrutiny of a small number of transcripts, it was decided to recruit a larger diverse sample as this is the first qualitative study of this population in the United Kingdom (as far as we know) and we wanted to gain an overview. : a review of qualitative interviews in IS research. These thirty, from which we draw our findings, were chosen for translation based on heterogeneity in depressive symptomology and educational attainment. Mixed method studies and articles reporting more than one qualitative method of data collection (e.g. Finally, the pre-determination of sample size on the basis of sampling requirements was stated by one article though this was not used to justify the number of interviews (SHI10). respondents Article (DOCX 12 kb), List of eligible articles included in the review (N=214). 2006;18(1):5982. Sandelowski M. Real qualitative researchers do not count: the use of numbers in qualitative research. Sampling for qualitative research. (SHI57). The size of the sample was largely determined by the availability of respondents and resources to complete the study. (SHI131). qualitative interviews entrepreneurial quantitative respondents competencies D#HtTZH+X$u & Constantinou CS, Georgiou M, Perdikogianni M. A comparative method for themes saturation (CoMeTS) in qualitative interviews. The determination of sample size on the basis of pragmatic considerations was the second most frequently invoked argument (9.6% of all justifications) appearing in all three journals. Methodological studies in this area aim to provide guidance about saturation and develop a practical application of processes that operationalise and evidence saturation. Ericksons [62] dimensions of evidentiary adequacy are useful here. Google Scholar. We recommend, firstly, that qualitative health researchers be more transparent about evaluations of their sample size sufficiency, situating these within broader and more encompassing assessments of data adequacy. Characterising and justifying sample size sufficiency in interview-based studies: systematic analysis of qualitative health research over a 15-year period. Soc Sci Med. Qualitative health research is an inter-disciplinary field that due to its affiliation with medical sciences, often faces views and positions reflective of a quantitative ethos. Qual Health Res. A further systematic analysis [32] of health education research over 20years demonstrated that interview-based studies averaged 104 participants (range 2 to 720 interviewees). In: Ritchie J, Lewis J, editors. Sample sizes of BMJ and SHI articles did not differ significantly from each other. Choosing a suitable sample size in qualitative research is an area of conceptual debate and practical uncertainty. 2030 interviews) and single case (i.e. Using this approach it was possible to more carefully explore childrens discourse about the I, agency, relationality and power in the thematic areas, revealing the subtle discursive variations described in this article. Chicago, IL: Aldine; 1967. As the peer-review process becomes increasingly public, future research could usefully examine how reporting around sample size sufficiency and data adequacy might be influenced by the interactions between authors and reviewers. Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. Recommendations for designing and reviewing qualitative research in psychology: promoting methodological integrity. Thus, participants may not be representative of couples that experience postnatal PTSD. J Comput Inform Syst. (BJHP31). Accessed 17 May 2018. In a theory-driven content analysis, Francis et al. That sample size principles, guidelines and tools have been developed to enable researchers to set, and justify the acceptability of, their sample size is an indication that the issue constitutes an important marker of the quality of qualitative research. The discovery of grounded theory: strategies for qualitative research. One paper claimed theoretical saturation (BMJ06) conceived as being when no further recurring themes emerging from the analysis whilst another study argued that although the analytic categories were highly saturated, it was not possible to determine whether theoretical saturation had been achieved (BMJ04). quantitative easing We note, first, the limited number of health-related journals reviewed, so that only a snapshot of qualitative health research has been captured. The past decade has seen a growing appetite in qualitative research for an evidence-based approach to sample size determination and to evaluations of the sufficiency of sample size. BMJ. Qual Health Res. A minority of articles discussed how sample-related limitations (with the latter most often concerning the type of sample, rather than the size) limited generalizability. Additional File 2 lists all eligible studies that were included in the present analysis. This study focused on British Chinese carers of patients with affective disorders, using a qualitative methodology to synthesise the sociocultural representations of illness within this community. 2005;15(9):127788. 2014;11(1):2541. 2014;120:13541. Bowen GA. Naturalistic inquiry and the saturation concept: a research note. (SHI23). PLoS Med 2009; 6(7): e1000097. interview studies nested in a trial, as part of a broader ethnography, as part of a longitudinal research) were included if they reported only single-time qualitative interviews. 0000001056 00000 n Nevertheless, this more formulaic approach raised criticisms relating to assumptions about the conceptual [43] and ontological status of themes [44] and the linearity ascribed to the processes of sampling, data collection and data analysis [45]. 3) that needs to be further conducted, the analysis of which will not yield any new themes or ideas. Sandelowski M. One is the liveliest number: the case orientation of qualitative research. Four BJHP studies, all adopting IPA, expressed the appropriateness or desirability of small sample sizes (BJHP41; BJHP45) or hastened to explain why they included a larger than typical sample size (BJHP32; BJHP47). 2000;10(1):35. Others investigated PhD qualitative studies [31] and grounded theory studies [35]. We set a target of seven to 10 caregivers per site because of time constraints and the anticipated difficulty of accessing caregivers at some home based care services. Morse JM. The Abstract, Methods and Discussion (and/or Conclusion) sections of each article were examined by one author (KV) who extracted all the relevant information. 2016;18(1):47. The current study employed a sample of 10 in keeping with the aim of exploring each participants account (Smith et al., 1999). Given that older adults have been ignored in the research concerning suntanning, fair-skinned older adults are the most likely to experience skin cancer, and women privilege appearance over health when it comes to sunbathing practices, our study offers depth and richness of data in a demographic group much in need of research attention. In the BJHP, justifying articles (Mean rank=31.3) had significantly larger sample sizes than non-justifying studies (Mean rank=22.7; U=237.000, p<.05). Field Methods. Secondly, we invite researchers critically to consider how saturation parameters found in prior methodological studies and sample size community norms might best inform, and apply to, their own project and encourage that data adequacy is best appraised with reference to features that are intrinsic to the study at hand. (SHI127). (BJHP41). In the BMJ, one article (BMJ15) appealed to pragmatic reasons, relating to time constraints and the difficulty to access certain study populations, to justify the determination of its sample size. Despite increasing attention to rigor in qualitative research (e.g. 2017;27(4):591608. Finally, three SHI articles explained their sample size with reference to practical aspects: time constraints and project manageability (SHI56), limited availability of respondents and project resources (SHI131), and time constraints (SHI113). In total, 214 articles 21 in the BMJ, 53 in the BJHP and 140 in the SHI were eligible for inclusion in the review. In: Wittrock M, editor. Pragmatic considerations, such as resource constraints or participant response rate and availability, was the second most frequently used argument accounting for approximately 10% of justifications and another 23% of justifications also represented intrinsic-to-the-study characteristics (i.e. households satisfaction Are we there yet? Only one article (SHI30) used the term transferability to argue for the potential of wider relevance of the results which was thought to be more the product of the composition of the sample (i.e.
