What is the Binomial Test?

The Binomial Test is a test between two categories and it was the results of some of Daniel Bernoulli’s Experiments. What this test does is compare the calculated result against a Binomial Distribution. That’s about as helpful as someone repeating themseves with a term you’ve never heard before - like I just did - so let’s go a bit deeper.

A Bernoulli Trial is where you have clearly two unique and competing events: A coin flip is the canonical example. You then observe which event occurs and record that. So, you flip a coin and observe whether it lands on heads or tails and then you record what that result was. When you repeat this and collect multiple results, then this is called the Bernoulli Process because you are repeating the same Bernoulli Trial. Once all the results are collected - and since the events are independent of one another -, you get a distribution that tells you the probability that event one or event two will occur over the n flips - or trials. When you give the Binoimial Test the values, it then tells you how often you could expect to see the result and compares that to a true Binomial Distribution.

The Binomial Test in Python

There is a function for this in the scipy package which we’ll use one more. And again, we’ll use the data which was used in the previous post about the Chi Squared Test.

# Binomial Test in python:
import scipy as sci
import pandas as pd

data = pd.read_csv("https://query.data.world/s/ccu3fx6fivcq2vdyavd26fhzk5tokq")
data.head()

   Subject Pref
0        1    B
1        2    B
2        3    B
3        4    B
4        5    B

We’re going to use the same trick with the pivot table I used before to get this into the right format for the test. You can run a Binomial Test in python using the function *scipy.stats.binom_test()`:

sci.stats.binom_test( data.pivot_table( index = 'Pref', aggfunc=len ))[0]

4.223704562267977e-05

<string>:1: DeprecationWarning: 'binom_test' is deprecated in favour of 'binomtest' from version 1.7.0 and will be removed in Scipy 1.12.0.

What I did above is the old method - which I was using and was included in my notes. I considered not including the older code but:

Someone else might run across this problem and this may be helpful.
This is a good time to show a bit more about how this function works.

There are some differences already between the two function’s expected inputs:

from inspect import signature
signature(sci.stats.binom_test)

<Signature (x, n=None, p=0.5, alternative='two-sided')>

signature(sci.stats.binomtest)

<Signature (k, n, p=0.5, alternative='two-sided')>

If we look at the documentation page we can see that this no longer allows for passing an array or dataframe. Instead, it expects us to pass integers. So, what are the parameters Number of Successes and Number of Trials? As per my previous discussion above, those would be how often the coin was heads and how many times the coin was flipped. But, since this data was about user preferences on a website then they would be how often subjects had a preference for A and how many records there are in our data!

xtab = data.pivot_table( index = 'Pref', aggfunc=len )
rst = sci.stats.binomtest(xtab.iloc[0,0], n=len(data))
print(f"The Binom Test Statistic is {round(rst.statistic, 3)} with a p-value of {rst.pvalue:.5E}")

The Binom Test Statistic is 0.233 with a p-value of 4.22370E-05

So, from these results we reject the null hypothesis that these two possibilites are equally likely to occur. That was quite a bit of work and we’re using a new function so how do we know we did not make a mistake? We know we got the same answer as the old version of the function so that is a good sign. And, we’ll be verfiying it again with R next.

Binomial Test in R

R’s version is so much easier to use. We’ll take the code from before to get the data downloaded.

library(tidyverse)

── Attaching packages ─────────────────────────────────────── tidyverse 1.3.2 ──
✔ ggplot2 3.4.0      ✔ purrr   1.0.1 
✔ tibble  3.1.8      ✔ dplyr   1.0.10
✔ tidyr   1.3.0      ✔ stringr 1.5.0 
✔ readr   2.1.3      ✔ forcats 0.5.2 
── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::filter() masks stats::filter()
✖ dplyr::lag()    masks stats::lag()

df = read_csv(
    "https://query.data.world/s/ccu3fx6fivcq2vdyavd26fhzk5tokq",
    col_types='if' # specify the column types are integer, factor.
)
df %>% head

# A tibble: 6 × 2
  Subject Pref 
    <int> <fct>
1       1 B    
2       2 B    
3       3 B    
4       4 B    
5       5 B    
6       6 B

… and we’ll just send it right on to the function:

df %>%
    # format it as expected
    xtabs(~ Pref, .) %>%
    # pipe into the test for the results.
    binom.test


    Exact binomial test

data:  .
number of successes = 46, number of trials = 60, p-value = 4.224e-05
alternative hypothesis: true probability of success is not equal to 0.5
95 percent confidence interval:
 0.6396172 0.8661627
sample estimates:
probability of success 
             0.7666667

There we go! So much easier to do this in R than it is in python. You can just tell that this language was designed with Statistics in mind with how much easier it is to get results.