McNemar's Test Calculator

Omni's McNemar's test calculator can help you perform different versions of McNemar's test for paired proportions - as many as five versions of this test are available. In particular, this tool can serve as McNemar's exact test calculator.

Not sure what the definition of McNemar's test is? Scroll down to learn everything you need about it (the correct interpretation of McNemar's test included). At the end of this article, you'll find an example of McNemar's test to see how it works in practice.

Let's first discuss what the assumptions of McNemar's test are. As you might've already heard, this statistical test deals with paired binomial data. More precisely, we call on McNemar's test when we have two paired groups, for instance, patients before and after treatment. Additionally, we have the values of a dichotomous variable (i.e., a variable with only two categories). So, for our patients, we can have a result of some medical test that is either positive or negative.

Other examples of when we can use McNemar's test:

• Maths exam result (pass vs. fail) of a group of students prior to and following a special preparatory curse.
• The will of citizens (yes vs. no) to vote for candidates before and after watching their TV spot.

Like in the t-test, the crucial assumption of McNemar's test is that we have a simple random sample (i.e., a set of independent, identically distributed random variables).

When we want to perform McNemar's test, the first step is to prepare the contingency table. We'll discuss this in the next section. And once we have the contingency tables, we'll be able to state the hypotheses of McNemar's test more formally. Let's go!

The contingency table that appears in the context of McNemar's test is a 2 x 2 table registering how many subjects fall into a given pair of categories. For instance, for patients subject to a test before and after treatment, we have the following pairs of categories:

• Before treatment and positive test result;
• After treatment and positive test result;
• Before treatment and negative test result; and
• After treatment and negative test result.

The corresponding contingency table looks like this:

As you can see, there are four fields, which we need to fill in with the data we obtained during our experiment. Assume we have:

• a people in the before-and-positive group;
• b people in the after-and-positive group;
• c people in the before-and-negative group; and
• d people in the after-and-negative group.

This translates into the following table:

We can now easily add to our table both column totals and row totals. By N, we denote the total number of observations.

Having discussed the contingency table, we are in a good position to discuss the hypotheses connected to McNemar's test.

Let's denote by p_a, p_b, p_c, p_d the theoretical probability of their respective group:

As probabilities always sum to one, we obviously have:

p_a + p_b + p_c + p_d = 1

Now, the null hypothesis of McNemar's test is a claim about the marginal distributions. Namely, it states that the row marginal and column marginal are equal - people often call this the hypothesis of marginal homogeneity:

H₀: p_a + p_b = p_a + p_c

(and p_c + p_d = p_b + p_d, but this follows immedately from the previous equation).

The alternative hypothesis is that the marginal distributions are not equal, i.e., that the above equation does not hold:

H₁: p_a + p_b ≠ p_a + p_c

Simplifying these equations, we get a slightly simpler form of our hypotheses, which you can often see in textbooks:

H₀: p_b = p_c

H₁: p_b ≠ p_c

McNemar's test interpretation

Let's discuss the interpretation of McNemar's test by stating the hypotheses this test deals with, without any mathematical notation:

• The null hypothesis of McNemar's test is that the treatment (or the preparatory math course or the TV spot) has no effect on the proportion of people with the positive result (or, respectively, the proportion of students passing the exam, or that of citizens willing to vote in favor of our candidate).
• The alternative hypothesis is that these proportions differ significantly, and so the treatment (course/TV spot) does have some effect.

A quick tip: don't focus that much on formulas, but try to remember the examples - they are beneficial when it comes to recalling the correct interpretation of McNemar's test for paired proportions.

When we've got the contingency table ready, we can quickly and easily compute McNemar's test statistic. The formula employs only the off-diagonal fields from the contingency table:

χ² = (b - c)² / (b + c)

Under the null hypothesis and if b and c are large enough, then χ² follows approximately the chi-squared distribution with 1 degree of freedom.

Note that, under the null hypothesis, we expect to observe small test statistic values. Indeed, if p_b = p_c, then in our data b should be roughly equal to c, which translates into small values of χ². Hence, by contradiction, big values of χ² provide evidence in favor of the alternative hypothesis H₁. The next step of McNemar's test depends on the approach you want to use:

• p-value approach - To calculate the p-value, you have to compute the probability that a random variable with the chi-squared distribution with 1 degree of freedom assumes values greater than or equal to the value of the test statistic χ² we've got for our sample:

  p-value = 1 - cdf(χ²)

  where cdf is the cumulative distribution function of the chi-squared distribution with 1 degree of freedom. If the p-value is smaller than the significance level we set for the test, we reject the null hypothesis.

• Critical region approach — The critical region is right-sided, i.e., it takes the form [k, ∞), where the critical value k is the (1-α)-quantile of the chi-squared distribution with 1 degree of freedom and α is the significance level you set for the test. If χ² falls into the critical region, i.e., if χ² ≥ k, we reject the null hypothesis.

Corrections to the test statistics

When we approximate a smooth distribution (in this case, the chi-squared distribution) with a discrete distribution, we are bound to have some inaccuracy, for which we can compensate by using a continuity correction. In the case of McNemar's test, the correction appears in the numerator of the test statistics. It involves:

1. Taking first the absolute value of b-c.
2. Subtracting some constant from it.
3. Squaring the resulting number.

The denominator remains unchanged. There are two corrections you can encounter:

• Edwards correction

  χ² = (|b - c| - 1)² / (b + c)

• Yates correction

  χ² = (|b - c| - ½)² / (b + c)

Edwards correction is more widespread, and some statistical packages use it by default - keep that in mind when they give a different value of χ² than you expected.

As mentioned earlier, in the standard way of performing McNemar's test, we approximate the distribution of the test statistic with the chi-squared distribution with 1 degree of freedom. However, as always, this approximation fails when our sample contains too few observations.

In such a case, we do not approximate but rather perform the exact binomial test. Here's the formula for its (two-sided) p-value:

After a moment's thought, one can recognize here the cumulative distribution function of the binomial distribution b(n,p) with parameters p = 0.5 and n = b + c. We use the exact McNemar's test if our sample satisfies b + c < 25.

Alternatively, we can use the mid-p-value McNemar's test, which we can calculate by subtracting the probability of the observed b from the exact two-sided p-value, that is:

Here's the full formula:

The mid-p-value approach has been recently becoming more and more popular.
We won't throw at you any technical details; let us just say that the mid-p-value allows us to avoid the disadvantages of the other methods (especially those related to controlling the type I error) while keeping their advantages to the almost full extent. In Omni's McNemar's exact test calculator, we've implemented both exact-p-value and mid-p-value so that you can easily compare the results these two approaches yield.

To perform McNemar's test with the help of our calculator, follow these steps:

1. Input your data (that is, the coefficients of the 2x2 contingency table) into the dedicated fields. The filled-in contingency table will appear at the bottom of McNemar's test calculator along with its marginals so that you can easily verify if you've entered data correctly.

2. Decide on the version of the McNemar's test you want to perform. There are five options available:

   • Standard McNemar's test;
   • With Edwards correction;
   • With Yates correction;
   • Exact binomial test; and
   • Mid-p binomial test.

   If you are not sure which one to choose, go for the standard McNemar's test.

3. Our McNemar's test calculator will return the p-value, based on which you can decide if your findings are significant at the desired significance level.

4. Additionally, for tests based on the χ² test statistic, our McNemar's test calculator displays the value of test statistic so that you can easily verify your own calculations.

In this final section, we'll go through an example of McNemar's test for paired proportions together to see how all these calculations work in practice.

Assume we have the following data:

1. Calculate the value of test statistics:

   χ² = (b - c)² / (b + c) = (70 - 50)² / (70 + 50) = 400 / 120 = 3.33

2. Determine the value of cdf of the chi-squared distribution with 1 degree of freedom at 3.33:

   cdf(3.33) = 0.932

3. Determine the p-value:

   p-value = 1 - cdf(3.33) = 0.068

As we can see, at the standard significance level 0.05, we do not have enough evidence to reject the null hypothesis - our findings are not statistically significant.

That's it, we're done with our example of McNemar's test! We hope you agree that's it's not at all hard to perform this test even if you have to compute it by hand. But, certainly, it's much more fun to use Omni's McNemar's test calculator 😉.