What to Do When the Committee Questions Your Sample Size
The situation you are facing
You have just presented your methods or results when a committee member asks, “Why did you choose this sample size?”, “What supports the claim that the number of observations is sufficient?”, or “Does your sample represent the population?” At that point, separate the question into three parts: which rule you used to choose the sample size, how that rule fits your model, and whether the actual data meet the requirements of your analysis.
You cannot justify sample size with a single number sitting in a .sav file. Open your proposal, questionnaire, number of items, number of independent variables in the regression, or SmartPLS model again. If your current answer is only “I referred to a previous thesis” or “This is how many responses I received,” the committee has a reason to ask further questions.
You can also review the guidance in Scientific research, but when your defense is approaching, start with the exact basis already used in your thesis.
Common reasons this happens
There are four common reasons why the sample-size justification attracts questions. First, the thesis uses several rules without identifying which one is the primary basis. For example, Chapter 3 mentions both the Yamane formula and the rule of 5 to 10 observations per item, then concludes that the collected responses are sufficient without showing the calculation.
Second, the minimum sample size was calculated before the questionnaire was finalized. When you add items, add analysis groups, or change from regression to a model with a mediator, the original number may no longer fit the design.
Third, you confuse the number of questionnaires distributed with the number of valid observations. The committee usually cares about the rows of data actually entered into Cronbach's Alpha, EFA, regression, or SmartPLS. Responses with many missing items, one-column answering patterns, or failed screening questions should not be counted in the analysis sample.
Fourth, you use the phrase “representative sample” too broadly. With convenience sampling, describe the respondents, location, period, and screening criteria clearly. You can say that the sample fits the defined research scope, but you should not claim that it represents the entire population when the sampling design does not support that conclusion.
The following is illustrative output.
| Sign when you are questioned | Possible reason | Check immediately in your records | Recommended response |
|---|---|---|---|
| You remember only the total number of questionnaires distributed | You have not separated valid and excluded responses | Compare the data file with the cleaning record | State separately the number received, the number excluded, and the final number of observations |
| Several sample-size formulas appear in the thesis | You have not identified the primary basis | Re-read Chapter 3 and the methods section | Select one primary basis and use the others only for comparison |
| The sample is smaller than five times the number of items | You applied the observations-per-item rule without calculating it correctly | Count the items actually used in the analysis | Show the calculation and explain how you will proceed if the requirement is not met |
| The model has many independent variables | You calculated only from the questionnaire and ignored the model | Count m in the regression model | Check the rule n of at least 50 + 8m (Tabachnick and Fidell, 2013) |
| You ran SmartPLS but presented a CB-SEM basis | You confused the method families | Check the software and model type | Explain the sample basis for PLS-SEM and do not include CFI, TLI, or RMSEA in this part |
| Many data rows were removed during cleaning | You planned only for the response-exclusion rate | Compare the original file with the analysis file | Report the final number of observations and the reason for exclusion transparently |
For regression, a commonly used basis is a minimum number of observations equal to 50 plus 8 times the number of independent variables, according to (Tabachnick and Fidell, 2013). For exploratory scale research, the rule of 5 to 10 observations per item is cited from (Hair et al., 2010). These calculations answer different questions, so explain which one fits your design.
How to handle the issue in priority order
Identify the number of observations actually analyzed
Open the data file and record three numbers: the number of responses received, the number remaining after cleaning, and the number of observations actually entered into each analysis. These numbers can differ. For example, some rows may be removed before EFA but remain in the original file, or one group of variables may be run on a different number of rows because of missing values.
In SPSS, check the number of cases through Analyze > Descriptive Statistics > Frequencies, selecting an ID variable or a demographic variable with few missing values. If the data include response codes, also check them through Analyze > Descriptive Statistics > Descriptives. Do not change the data merely to reach a desired threshold. Save one original data file and one analysis file, and record the reason for removing each row.
Compare the sample size with the number of items
Count the items actually used, such as Q1 through Q5 for one scale, rather than counting the constructs in the model. If you have 30 items and use the rule of 5 to 10 observations per item, the reference range is 150 to 300 observations. This rule is associated with (Hair et al., 2010), but you should not present it as a law that applies to every design.
If the sample is below the expected level, check whether you counted ID variables, screening questions, or variables removed before analysis. Conversely, if you used 30 items in EFA, calculating from only the original 20 items makes the justification inconsistent.
Check the number of independent variables in the regression
If your thesis uses linear regression, define m as the number of independent variables appearing in the equation. For example, a model with 5 independent variables gives a minimum of 90 observations under the 50 + 8m basis, according to (Tabachnick and Fidell, 2013). This is an additional check, not a substitute for considering the number of items and the research objective.
When you open the Coefficients table in SPSS, count the rows for the independent variables and exclude the Constant row. If the model includes control variables, state whether they are included in m and use that decision consistently in Chapter 3.
Check data quality before asking for more responses
A larger sample cannot repair poor-quality data. Check the missing-value rate, responses completed unusually quickly, respondents selecting the same option for almost the entire questionnaire, and duplicate response codes. For regression data, you can review the diagnostics under Analyze > Regression > Linear, including Collinearity diagnostics and Durbin-Watson. A Durbin-Watson value between 1 and 3 is commonly used when checking residuals, according to (Field, 2013).
If the sample is insufficient but the existing data are clean, ask your supervisor whether collecting more responses is possible. If the sample is large enough but contains many questionable rows, prioritize cleaning and explaining the exclusion criteria instead of keeping data merely to defend a number.
Match the basis to the analysis type
With SmartPLS, distinguish between the measurement model and the structural model. Explain sample size together with the number of latent variables, number of paths, and testing objective rather than copying the CFI, TLI, or RMSEA criteria from CB-SEM. Those fit indices belong to CB-SEM under (Hu and Bentler, 1999), so they are not the criteria to present for SmartPLS results.
If the committee asks about the stability of the results, say that you used bootstrapping and state the number of bootstrap subsamples if that is what appears in your analysis file. The procedure of 5,000 bootstrap subsamples is presented in (Hair et al., 2022). State this number only if you actually configured and ran it in SmartPLS.
When you need to collect more data
Consider collecting more data when the number of valid observations is below the minimum basis committed to in the thesis, when one analysis group has too few cases, or when the sample size drops substantially after invalid data are removed. Base this decision on the current data file, not on the feeling that “a few more responses would make it look better.”
The table below provides a quick comparison of common bases. Each row includes a source so that you do not answer with a threshold whose origin is unclear.
| Check basis | Threshold or calculation | Canonical source |
|---|---|---|
| Multiple regression | n of at least 50 + 8m, where m is the number of independent variables | (Tabachnick and Fidell, 2013) |
| Items in exploratory research | 5 to 10 observations per item | (Hair et al., 2010) |
| Factor analysis | Also assess sample adequacy using KMO of at least 0.5 | (Kaiser, 1974) |
| Finite population sample size | Apply the finite-population sample-size formula selected in the proposal | (Yamane, 1967) |
| Factor analysis adequacy | Consider factor-loading quality and factor structure together, not sample size alone | (Comrey and Lee, 1992) |
If you can collect more responses, keep the screening criteria, questionnaire distribution method, and variable structure unchanged. Do not change the respondent group between the two rounds without documenting it, because the committee may ask whether the two groups belong to the same research population.
The results tables themselves may also point to a problem. For example, a KMO and Bartlett's Test table with a KMO below the reference level, or a Rotated Component Matrix with many cross-loadings, does not mean that increasing the sample alone will solve everything. The issue may be the wording of the items, the scale structure, or the suitability of the data for the model.
How to explain it to your supervisor
When you discuss the issue, bring a one-page summary showing the target population, sampling method, number of questionnaires distributed, number of valid responses, number of items, number of independent variables, and sample-size basis. This lets your supervisor see the complete logic instead of hearing one isolated number.
How to write this in your thesis: “The study received [number of responses] responses. After excluding [number of excluded responses] invalid responses, [number of valid observations] valid observations remained for analysis. The sample size was assessed using [primary basis], where [m is the number of independent variables or number of items] and the minimum requirement was [threshold]. Therefore, the actual sample size [meets or does not meet] the selected basis. The conclusions are limited to [population, location, and study period].”
This is a template for replacing the placeholders with your actual figures. Do not copy it unchanged if the information in your file is different. If the sample size is insufficient, write the limitation honestly, explain its possible effect on generalizability, and ask whether you should collect more data or narrow the scope of the conclusions.
When the committee asks directly, answer in this order: “I used [basis] as the primary basis because the research objective is [regression, EFA, or PLS-SEM]. With [number of variables], the minimum is calculated as [calculation]. After cleaning, [number of observations] valid observations remained. Therefore, I limit the conclusions to [scope] and do not claim that the sample represents the entire population beyond that scope.”
How to prevent the problem during design
As soon as you design the questionnaire, create a variable tracking table. Each row should include the variable code, construct, scale source, number of items, coding method, and planned analysis. When the number of items changes, you will immediately see the effect on sample size.
Select the sample-size basis before distributing the questionnaire and record a contingency plan. If you need 200 valid observations but expect 10% of responses to be excluded, the number of responses to collect must be higher than 200. Describe this contingency percentage as a data-collection plan, not as a claim that every distributed questionnaire will become an analyzed observation.
Agree on the exclusion criteria before viewing the results. For example, define what counts as a questionnaire with too many missing items, a duplicate response, or a failed screening question. Setting the criteria after seeing a low Cronbach's Alpha can make the committee suspect that you removed data to obtain the desired result.
If the model includes comparison groups, a mediator, or a moderator, account for the final analysis from the start. A sample that is sufficient for an overall regression may still be insufficient for smaller groups. This is why you should map the analysis from the research question to the output table instead of calculating sample size only from the number of questionnaire pages.
Common mistakes during revision
The first mistake is increasing the number of rows by duplicating responses. Such data distort the standard deviation, correlations, and p-values, and create a serious risk when the committee asks to open the file or check response codes. If your data authenticity is being questioned, review what to do when your data are suspected of being fabricated and handle the issue transparently.
The second mistake is deleting variables or responses merely to reach the required sample size or make the results table look better. Every exclusion needs a criterion, a record, and an explanation that was possible before the final results were known.
The third mistake is using one rule to answer every question. The rule of 5 to 10 observations per item does not automatically justify a multiple regression model, and a finite-population formula does not automatically prove that a convenience sample is representative.
The fourth mistake is forcing your data to resemble the results of the original article. If the sample size, context, or sampling method differs, the results may differ as well. When this happens, you can read what to do when your results differ from the original article.
Finally, do not prepare only a spoken answer. Have the data file, syntax or sample-size spreadsheet, and the Case Processing Summary table open if available. For a question specifically about sample size in SPSS, committee questions about SPSS sample size can help you check the presentation as well.
Read more: scientific research.
Frequently asked questions
How should I answer if the committee asks what sample size is sufficient?
Do not answer with one general number for every thesis. State the analysis type, the variables included in the calculation, the selected basis, and the number of valid observations after cleaning. For multiple regression, you can show the 50 + 8m calculation from (Tabachnick and Fidell, 2013) if that is the basis used in your thesis.
Do I need to redo everything if I collected fewer responses than planned?
Not necessarily. First check the number of valid observations, the number of items, and the final analysis. If the sample still meets the committed basis, you can keep it and state the limitation. If it does not, discuss collecting more data or adjusting the scope of the conclusions with your supervisor, and do not alter the data yourself.
Do I need to collect more data if the sample is sufficient but KMO or EFA is not acceptable?
Additional data may make the results more stable, but sample size is not the only possible cause. Check item quality, cross-loadings, reverse-coded variables, and scale suitability. A KMO of at least 0.5 is a reference level according to (Kaiser, 1974), but decisions about retaining or removing items require a review of the complete EFA structure.
Can I use several sample-size formulas in the same thesis?
You can use one formula as the primary basis and one or two rules for comparison, provided that you explain the role of each. If you include several formulas without explaining why you selected the final number, the methods section may look pieced together.
Can a convenience sample be called representative?
Avoid claiming absolute representativeness when you do not have an appropriate probability sampling design. Describe the respondents, screening criteria, location, period, and scope reflected by the data. The phrase “appropriate for the research scope” is usually more accurate when you use convenience sampling.
Before your defense, open the data file and fill in the placeholders in the justification paragraph: [number of responses], [number of excluded responses], [number of valid observations], [number of variables], and [primary basis]. If you need to rerun the analysis on the actual .sav or .csv file, you can use DoThesis M4 data analysis.