4.1 The Null (and Alternative) Hypothesis

A null hypothesis \(H_0 : \theta \in \Theta_0\) is a statement about the value of a parameter. A null hypothesis states that the parameter \(\theta\) takes on a value from particular subset \(\Theta_0\) of the parameter space \(Theta\).

Typically, the null hypothesis includes the parameter values that the researcher would like to argue against.

A hypothesis test is a rule that uses a sample to decide whether to reject or not reject the null hypothesis.

A hypothesis testing formal procedure that researcher use to assess the evidence against the null hypothesis. In the context of theory testing, you can think if the null hypothesis as the hypothesis that the theory is wrong or that the parameters of interest are insistent with the theory.

Importantly, the result of the hypothesis test will be to either reject (i.e., deem false) or not reject (i.e., not deem false) the null hypothesis. Crucially, when not rejecting the null hypothesis, the researcher does not take the additional step of claiming that the null hypothesis is correct. Instead, they remain agnostic about whether the null hypothesis is true.

To emphasize this point, Tukey and Jones (2000) write:

With this formulation, a conclusion is in error only when it is “a reversal,” when it asserts one direction while the (unknown) truth is the other direction. Asserting that the direction is not yet established may constitute a wasted opportunity, but it is not an error. We want to control the rate of error, the reversal rate, while minimizing wasted opportunity, that is, while minimizing indefinite results.

Each null hypothesis \(H_0 : \theta \in \Theta_0\) implies an opposite hypothesis \(H_1 : \theta \in \Theta_0^C\) called the alternative hypothesis. Rejecting the null hypothesis is equivalent to accepting the alternative hypothesis. When the researcher does not reject the null hypothesis, they remain agnostic about whether the null hypothesis is true; therefore, they must also remain agnostic about whether the alternative hypothesis is true.

Because the the null and the alternative hypotheses are exactly opposites, I leave the alternative as implicit.

A point null hypothesis has the form \(H_0 : \theta \in \theta_0\), where \(\theta_0\) represents a single point in the parameter space \(\Theta\).

A composite null hypothesis has the form \(H_0 : \theta \in \Theta_0\), where \(\Theta_0\) represents multiple points in the parameter space \(\Theta\).

For example, the null hypothesis \(H_0 : \theta \leq 0\) is a composite null hypothesis.

4.1.1 The Most Common Null Hypotheses

In political science, the most common hypotheses are about differences between the averages of two groups or about slopes in regression model.

  • non-directional null hypothesis for an experiment: \(\mu_{\text{treatment}} - \mu_{\text{control}} = 0\).
  • directional null hypothesis for an experiment: \(\mu_{\text{treatment}} - \mu_{\text{control}} \leq 0\) or \(\mu_{\text{treatment}} - \mu_{\text{control}} \geq 0\).
  • non-directional null hypothesis for a regression slope: \(\beta_x = 0\).
  • directional null hypothesis for a regression slope: \(\beta_x \leq 0\) or \(\beta_x\geq 0\).

4.1.2 Errors

For each test, there are two possible states: either the null hypothesis is true or it is false. Similarly, there are two possible outcomes: either the researcher rejects they null or they do not.

If the null hypothesis is true, then:

  • The researcher might (incorrect) reject it, in which case they have made an error. This is (for reasons I don’t understand) called a Type I error.
  • The researcher might not reject it, in which case they made no error.

If the null hypothesis is false, then:

  • The researcher might (correctly) reject it, in which case they made no error. This is the most desired outcome.
  • The researcher might not reject it. The null hypothesis is false, and the researcher did not reject it. However, the researcher (if they followed my advice above) did remain agnostic. So it’s not quite an “error”–it’s more of a “lost opportunity.” (The researcher had the correct suspicion, but the data were inconclusive.) This is called a Type II error.