What the results mean

When you ask respondents to rate two alternatives on such ratings as like/ dislike each alternative or have a problem/ don’t have a problem with each alternative, the McNemar Test can be used to test for significant differences in the proportions. In particular, a researcher would like to know if one product obtains a “like” rating more than the other or if an alternative is significantly more likely to experience a problem than the other alternative. There is some likelihood (confidence level) that the study results will show the proportion of responses for one product to be significantly different than the other product. This test takes into account the correlation that exists because a respondent could rate positively alternative 1 only, alternative 2 only or they could rate both alternatives positively or neither alternative positively. If the Z-value obtained from this test is greater than the value Z-Value for a given confidence level, the proportions are said to be significantly different.

More information

Step 1 – Confidence Level

The value chosen in Step 1 determines the confidence level of your results. It tells you the likelihood that the difference in proportions is NOT due to random chance. For most marketing research studies, a confidence level of 95% is used.

Confidence level is related to the level of significance (α ). A 95% Confidence level corresponds to α = .05. If the Level of significance (α) = .05, that means that there is one chance in twenty that the true proportions are not significantly different.

Step 2 – Enter the total base population size

This number should be positive and represents the total number off respondents to be asked the two questions.

Step 3 – Enter the total respondents that selected choice “A”

This number should be positive and represent the number of respondents selecting the first alternative. This should represent the total number of responses “A” received without regard to what the response to alternative “B” was. You can enter either the frequency response or the percentage response. The other number will auto-calculate.

Step 4 – Enter the total respondents that selected choice “B”

Repeat Step 3 for choice “B.” This number should be positive and represent the number of respondents selecting the second alternative. This should represent the total number of responses “B” received without regard to what the response to alternative “A” was. You can enter either the frequency response or the percentage response. The other number will auto-calculate.

Step 5 – Enter number of respondents that selected neither “A” nor “B”

This number should be positive and represent the number of respondents that selected neither “A” nor “B.” You can enter either the frequency response or the percentage response. The other number will auto-calculate.

Click the calculator button to obtain the results of the test. In addition to the statistical significance results, certain ther numbers (e.g. the number of respondents selecting each alternative only as well as the number selecting boith alternatives) will auto-calculate.

Assumptions

This test assumes that your sample size is 30 or larger.

The calculator on this page includes the continuity correction which compensates for the discontinuity of the discrete binomial distribution. This correction is recommended when sample sizes are only modestly large but it is acceptable to use this correction for all sample sizes.

This calculator calculates both one-tail and two-tailed tests. If you are testing the null hypothesis that the two alternatives are equal, use the two-tailed test result.