---
title: "Discussion section 4"
subtitle: "Probability and sampling"
author: "30 September 2026"
format: pdf
---


# Problem 1: Error rates in hypothesis testing

You have a fancy device that tests null hypotheses. Null hypotheses are statements about the world that can be either true or false. The device is designed to turn red when a null hypothesis is false and green when it is true, but it doesn't work perfectly: when a null hypothesis is false it turns red with probability 4/5 (i.e. it mistakenly turns green with probability 1/5), and when a null hypothesis is true it turns green with probability 19/20 (i.e. it mistakenly turns red with probability 1/20). Tests of different null hypotheses are independent, and 9/10 of the null hypotheses you test are true.

**(1a)** If you test 10 true null hypotheses in a row, what is the probability that the alarm turns red at least once? Explain your solution with reference to any axioms/definitions/rules/laws of probability you use.

**Answer**:

Call the event of rejecting at least one null hypothesis $A$. That's the probability we want to know. There are many ways for $A$ to occur -- e.g. reject every time, reject the first null hypothesis but not thereafter, etc -- and it may be tedious to list them and compute the probability of each one. But there is only one way for $A^C$, the complement of $A$, to occur: on every test, the light turns green. The complement rule says that $P(A) = 1 - P(A^C)$. So what is $P(A^C)$? Given that the null hypothesis is true, the probability of the light turning green a single test is given as 19/20. The tests are independent, so by the definition of independence the probability of ten green lights in a row is $(19/20)^{10}$. Therefore the probability of the light turning red at least once is $1 - (19/20)^{10} \approx .401$.

```{r}
1 - (1 - .05)^10
```

**(1b)** Write a simulation to check your answer to (1a). That is, use R to generate many draws according to the random process described (testing ten true null hypotheses in a row), and confirm that the proportion of draws with at least one red light is approximately the same as in your answer above.

```{r}
# using the rbinom function, which gives us n results of the experiment,
# i.e. n counts of red lights in each experiment

tests <- rbinom(n = 10000000, size = 10, prob = .05)
mean(tests > 0)
```

Using "sample()"

```{r}
# can also do it using sample
# here is a large number of tests -- 1 means red light when null is true

red_lights <- sample(x = c(1, 0), size = 1000000, 
                     prob = c(.05, .95), replace = T)

# we organize them in a big matrix with 10 columns,
# so that each row is a set of 10 tests

reshaped_red_lights <- matrix(red_lights, ncol = 10)

# we use apply() to get the sum in each row --
# the number of red lights in 10 tests

reds_per_row <- apply(reshaped_red_lights, 1, sum)
mean(reds_per_row > 0)
```

**(1c)** What is the probability of getting a red light in any given test (i.e. when you don't know if the null hypothesis is true)? Explain your solution with reference to any axioms/definitions/rules/laws of probability you use.

**Answer**: Let $R$ denote the event that the light turns red, and let $H_0$ denote the event that the null hypothesis is true (with $H_0^C$ denoting the event that the null hypothesis is false). Using the law of total probability, we have $P(R) = P(R \mid H_0)P(H_0) + P(R \mid H_0^C)P(H_0^C) = \frac{1}{20}\frac{9}{10} + \frac{4}{5}\frac{1}{10} = \frac{9}{200} + \frac{4}{50} = \frac{9}{200} + \frac{16}{200} = \frac{25}{200} = \frac{1}{8}$.

**(1d)** If the light turns red in a given test, what is the probability that the null hypothesis is false? Explain your solution with reference to any axioms/definitions/rules/laws of probability you use.

**Answer**: Here we use Bayes Rule, which can be derived from the definition of conditional probability or the product rule:

$$P(H_0^C \mid R) = \frac{P(R \mid H_0^C)P(H_0^C)}{P(R)}$$

Using our answer for $P(R)$ above, we have $(4/5)(1/10)/(1/8) = 32/50 = 16/25$.

**(1e)** Write a simulation to check your answer to (1c) and (1d). That is, use R to generate many draws according to the random process described (testing null hypotheses), and confirm your answer about the proportion of red-light-producing draws (1c) and the proportion of red-light-producing draws in which the null hypothesis is false (1d).

```{r}
# first a version with no randomness -- we make a joint distribution
# that exactly matches the conditions in the problem
ntrue <- 9000
nfalse <- 1000
null_is_true <- c(rep(T, ntrue), rep(F, nfalse))
red_light <- c(rep(T, ntrue*(1/20)), rep(F, ntrue*(19/20)),
               rep(T, nfalse*(4/5)), rep(F, nfalse*(1/5)))
# confirming 1c
mean(red_light)
# confirming 1d
mean(!null_is_true[red_light])
```

```{r}
# now a version with randomness
n <- 100000
null_is_true <- sample(c(F, T), size = n, replace = T, prob = c(.1, .9))
red_light <- rep(NA, n)
red_light[null_is_true] <- sample(c(F, T), size = sum(null_is_true),
                                  replace = T, prob = c(.95, .05))
red_light[!null_is_true] <- sample(c(F, T), size = sum(!null_is_true),
                                   replace = T, prob = c(.2, .8))
# 1c
mean(red_light)
# 1d
mean(!null_is_true[red_light])
```

# Problem 3: continuous random variables

**(3a)** Let $X$ be uniformly distributed between -5 and 3. Compute $\Pr[X < 1]$ and $\Pr[-3 < X < 1/2]$ analytically (e.g. by computing the length and height of the area to be integrated) and confirm your results using R.

**Answer**: The uniform density is a rectangle. In this case it goes between -5 and 3 so its length is 8. Its area must be 1 (this is true of any density), so its height must be 1/8.

$\Pr[X < 1]$ is the area of this rectangle below 1, so its length is 6; its area is therefore $6 \times 1/8 = 3/4$. Confirmation in R:

```{r}
punif(1, min = -5, max = 3)
```

$\Pr[-3 < X < 1/2]$ is the area of this rectangle between -3 and 1/2, so its length is 7/2; its area is therefore $7/2 \times 1/8 = 7/16$. Confirmation in R:

```{r}
punif(1/2, min = -5, max = 3) - punif(-3, min = -5, max = 3)
```

**(3b)** Let $X$ be normally distributed with mean 0 and standard deviation 1. Using R, compute $\Pr[-1.96 < X < 1.96]$.

```{r}
pnorm(1.96) - pnorm(-1.96)
```