--- title: "{{< var course.code >}}: Week 1 Lab — R from Zero to Real Data" subtitle: "{{< var course.institution >}} · {{< var course.semester >}}" format: html: embed-resources: true toc: true toc-depth: 3 number-sections: true execute: echo: true warning: false message: false --- Welcome to the first lab. Lecture is where the ideas live; lab is where you put your hands on real data, every single week. Today we go from a blank machine to loading the course's running dataset — a national opinion survey from 2012 — and doing your first real data manipulation on it. We will end on an unresolved problem, on purpose. Work through this document top to bottom, typing the code yourself rather than copy-pasting. Typing is slower, and that is the point: your fingers are learning a language. # Before lab: install R and Positron Do this before you arrive if you can; it takes about ten minutes. 1. **Install R** — the language itself — from CRAN: [https://cran.r-project.org](https://cran.r-project.org). Choose your operating system and install the latest release with the default options. You need version 4.2 or above. 2. **Install Positron** — the environment we write R in — from Posit: [https://positron.posit.co/download.html](https://positron.posit.co/download.html). Order matters: R first, then Positron. Positron is not R; it is a workshop built around R, and it does not install R for you. If you open Positron and see a console with a `>` prompt, both installations worked. If the console never starts a session, R is either missing or too old — install it, then reopen Positron. Positron already ships with Quarto, so do not install Quarto separately. # A quick tour of Positron Open Positron. You will see: - **Console** (bottom): where R actually runs. Type a command at the `>` prompt, press Enter, get an answer. - **Editor** (center, appears when you open a file): where scripts live — files of commands you keep. - **Variables** (right sidebar): the objects you have created this session. - **Plots** and **Help** (right sidebar, alongside Variables): your figures and the documentation. - **Explorer** (left sidebar): the files in the folder you have open. The one keystroke worth learning today is the **Command Palette** (**Ctrl+Shift+P**, Mac: **Cmd+Shift+P**), which searches every command Positron has by name. When you cannot find a menu item, search for it there. ## The console: R as a calculator Click in the console and try a few lines. R evaluates each and prints the result: ```{r calculator} 2 + 2 7 / 2 (1745 - 1517) / 1517 # e.g., a growth rate: (new - old) / old ``` The `[1]` at the front of each answer is R telling you the position of the first printed value — it matters when results have many values, as they soon will. You store results with the assignment arrow `<-` ("gets"): ```{r assignment} turnout <- c(0.62, 0.58, 0.55, 0.51, 0.47) # c() combines values into a vector turnout mean(turnout) ``` Three things just happened that you will do thousands of times this semester: you **combined** values into a vector with `c()`, you **named** the result with `<-`, and you **applied a function** — `mean()` — to it. Everything in R is a variation on this pattern. ## Scripts: why we never work in the console alone Anything you type in the console vanishes when you close Positron. So we work in **scripts**: *File → New File → R File*. Type commands in the script, put your cursor on a line, and press **Ctrl+Enter** (Mac: **Cmd+Return**) to send it to the console. Save the script with a name like `lab01.R`. ::: {.callout-note} ## Why scripts are the whole game The script *is* your analysis. If your script runs top to bottom on a fresh session and produces your results, your work is reproducible: you can rerun it, your co-author can rerun it, and reviewers can rerun it. If your results depend on things you clicked or typed once and forgot, they are not results — they are anecdotes. Every empirical claim you make in this course should trace back to a script. ::: Two console habits to build immediately: the **Up arrow** recalls previous commands, and `?mean` (a question mark before any function name) opens its help page. # Packages: installing the tidyverse Base R can do everything we need, but the **tidyverse** — a suite of packages with a consistent grammar for data work — is the dialect this course speaks. Install it once per machine: ```{r install} #| eval: false install.packages("tidyverse") # once per computer; note the quotes ``` Give it a few minutes. Then, in *every* R session where you want to use it: ```{r library} library(tidyverse) # once per session; no quotes needed ``` The distinction trips everyone up at first: `install.packages()` buys the book (once); `library()` takes it off the shelf (every session). When you see the error `could not find function "read_csv"`, the diagnosis is almost always a missing `library(tidyverse)` at the top of your script. When you load the tidyverse you will see a burst of startup messages listing the attached packages and a couple of "conflicts." That is normal, not an error. # Folders and relative paths Before touching the data, we set up the habit that prevents 90% of beginner misery: keeping everything for this course in **one folder**, and opening that folder in Positron. Positron treats whichever folder you open as home base. Make a folder for this course somewhere you can find it again — not in Downloads — and open it with *File → Open Folder*. Inside it, mirror the course layout: ``` gov6019/ <- the folder you open in Positron data/ <- voter2012.csv and its codebook scripts/ <- gov6019_ggplot.R (the course figure theme) weeks/ w01_intro_data/ <- this document, your lab01.R script ``` Everything in `data/` and `scripts/` downloads from the course site — grab `voter2012.csv`, its codebook, and `gov6019_ggplot.R` now. (The script becomes essential in Week 2, when we start making figures.) Why bother? Because of **paths**. Every file on your computer has an absolute path (`C:/Users/you/Desktop/gov6019/data/voter2012.csv` or `/Users/you/gov6019/data/voter2012.csv`) — and absolute paths are poison in a script, because they work on exactly one computer: yours, today. A **relative path** (`data/voter2012.csv`) works for anyone who has the folder. ::: {.callout-important} ## Never `setwd()` You will find old code online that starts with `setwd("C:/Users/someone/...")` to set the working directory by hand. Do not do this, ever. It hard-codes one person's machine into the script, which is exactly what we are trying to avoid. The folder you opened *is* your working directory; relative paths do the rest. This is a course rule and, increasingly, a discipline-wide norm. ::: One subtlety worth learning now, because it will otherwise bite you at a worse time: **where a relative path starts from depends on where the code runs.** Type `getwd()` in the console to see your current working directory. - Code you run in the **console** starts from the folder you opened: from there, the data is at `data/voter2012.csv`. - Code inside a **document like this one** starts from the document's own folder. This lab sits in `weeks/w01_intro_data/`, two levels below the course root, so it reaches the data by climbing up twice: `..` means "one folder up," hence `../../data/voter2012.csv`. Both are relative paths; they just start from different places. If R ever tells you a file does not exist, your first move is `getwd()`, not panic. # Loading the VOTER data The course's running dataset is `voter2012.csv`: a subset of the **2012 VOTER Study**, a large national opinion survey of U.S. adults (n = 8,000). We will live with this dataset for most of the semester — by Week 8 you will know its every quirk, and it has quirks. Load it with `read_csv()`: ```{r load} #| message: true voter <- read_csv("../../data/voter2012.csv") ``` ::: {.callout-tip} ## Which path goes in *your* script? Typing along in your own `lab01.R`, run from the console of your course folder? Then the path is `data/voter2012.csv` — the `../../` form above is what *this document* uses, because it starts from its own folder two levels down. This is the console-versus-document distinction from the previous section, live. ::: The message is worth reading, not skipping: 8,000 rows, 9 columns, and every column parsed as `dbl` (a number). Keep that last fact in mind. ## First looks Four inspection commands you will run on every dataset you ever meet: ```{r inspect-dim} dim(voter) # rows, columns names(voter) # variable names ``` ```{r inspect-glimpse} glimpse(voter) # every variable, its type, and its first values ``` ```{r inspect-head} head(voter) # the first six rows, as a table ``` Look at the actual rows. Every value is a bare integer. Is a `pid3` of `1` a Democrat or a Republican? Is `gender` `2` a woman or a man? Does `important_econ = 1` mean the economy is very important or not at all? **The data will not tell you.** This is how survey data actually arrives: the numbers live in the file and the meaning lives somewhere else. ## The codebook-first habit The meaning lives in the **codebook** — for us, `data/voter2012_codebook.md`. Open it now (Positron will display it: *File → Open File*), and keep it visible next to your script for the rest of this lab. Here is its variable table: | Variable | Question / meaning | Values | |---|---|---| | `important_econ` | "The economy is an important issue" (agreement) | 1 Strongly agree · 2 Agree · 3 Disagree · 4 Strongly disagree · 5 Don't know | | `important_gayr` | "Gay rights are an important issue" (agreement) | same scale | | `unimportant_budget` | "The budget deficit **is not** an important issue" (agreement) | same scale | | `unimportant_abort` | "Abortion **is not** an important issue" (agreement) | same scale | | `educ` | Highest education completed | 1 No HS · 2 HS grad · 3 Some college · 4 2-year · 5 4-year · 6 Post-grad | | `birthyr` | Year of birth | 1921–1994 | | `pol_interest` | Interest in politics and current affairs | 1 Not much · 2 Somewhat · 3 Very much · 4 Not sure | | `pid3` | Party identification | 1 Democrat · 2 Republican · 3 Other | | `gender` | Gender | 1 Man · 2 Woman | Empty cells in the CSV are item nonresponse — people who skipped the question — and R reads them as `NA`, its symbol for "missing." ::: {.callout-important} ## Rule: codebook before computation Never compute anything on a variable you have not looked up in the codebook. In this dataset the columns are numbers, but — think back to lecture — most of these variables are not *interval* measurements. `pid3` is nominal: its numbers are arbitrary labels. `educ` and the issue items are ordinal: order means something, distances do not. Only `birthyr` is a genuine number in the arithmetic sense. The codebook is what tells you which is which. ::: # Five verbs: your first real wrangling The tidyverse organizes data manipulation around a small set of verbs, each of which takes a dataset and returns a dataset. We will chain them with the **pipe**, written `|>`, which you can pronounce as "then": ```{r pipe-intro} voter |> dim() # exactly the same as dim(voter): take voter, THEN apply dim ``` The pipe earns its keep when steps stack up, which is most of the time. ## `select()`: choose columns ```{r select} voter |> select(pid3, gender, birthyr) |> head(3) ``` ## `filter()`: choose rows `filter()` keeps rows meeting a logical condition. Note the double equals `==` for "is equal to" (single `=` does something else entirely): ```{r filter} # Respondents who report being very interested in politics (codebook: 3) voter |> filter(pol_interest == 3) |> dim() ``` 5,063 of the 8,000 respondents. Conditions can be combined with commas (meaning "and"): ```{r filter-and} # Very interested AND identifying as Democrats voter |> filter(pol_interest == 3, pid3 == 1) |> dim() ``` ## `arrange()`: sort rows ```{r arrange} # Youngest respondents first: sort by birth year, descending voter |> arrange(desc(birthyr)) |> select(birthyr, pid3, gender, educ) |> head(3) ``` Drop the `desc()` and you sort ascending — you will need that in the exercises. ## `mutate()`: create new variables Here is the first of the codebook's little tests. There is no age variable — only `birthyr`. But the survey was fielded in 2012, so age is one subtraction away: ```{r mutate} voter <- voter |> mutate(age = 2012 - birthyr) # assign back: voter now HAS an age column voter |> select(birthyr, age) |> head(3) ``` Two things to notice. First, we assigned the result back to `voter` — without `voter <-` at the front, the new column is computed, printed, and thrown away. Second, this age is a good approximation, not an exact one: someone born in late 1994 who had not yet had their 2012 birthday is 17, not 18. For describing a dataset that is fine; the honest move is to know it and say it. ```{r age-range} range(voter$age) # $ pulls out a single column as a plain vector ``` Ages 18 to 91 — consistent with a survey of adults, which is exactly the kind of sanity check you should run on every variable you derive. ## `count()`: tabulate The workhorse for categorical variables: ```{r count-pid3} voter |> count(pid3) ``` Read this *with the codebook*: 3,639 Democrats, 3,091 Republicans, 1,181 identifying with neither major party — and 89 people with `NA`, who skipped the question. `count()` shows missing values as their own row rather than hiding them, which is the correct instinct and one we will keep. ```{r count-educ} voter |> count(educ) ``` Again: this table is unreadable without the codebook, and perfectly clear with it (`6` = post-grads: 1,210 of them, which will matter in the exercises). # A number that lies Time to compute our first substantive summary. The economy item asks for agreement with "The economy is an important issue." What was the average response in 2012, three years after the Great Recession? ```{r mean-na} mean(voter$important_econ) ``` `NA`. R is not broken; it is being scrupulously honest. Ninety-seven respondents skipped this question, so their values are missing — and R will not average numbers it does not have unless you explicitly tell it how to handle the gaps. The standard instruction is `na.rm = TRUE` ("remove `NA`s before computing"): ```{r mean-narm} mean(voter$important_econ, na.rm = TRUE) ``` There we go: **1.14**, on a scale where 1 = *Strongly agree*. In 2012, with unemployment near 8%, respondents all but unanimously agreed the economy was an important issue. Plausible. Publishable, even. Before we congratulate ourselves, one more habit: never summarize a variable you have not tabulated. What values does `important_econ` actually take? ```{r count-econ} voter |> count(important_econ) ``` Look at the fifth row. Thirty-nine people have the value **5** — on what the codebook describes as a four-point agreement scale. The economy: apparently so important that 39 respondents rated it a five out of four. Scroll back to the codebook table. The fifth value is right there: **5 = Don't know.** Now think about what our mean just did. It treated "Don't know" as a *substantive* answer — one notch beyond *Strongly disagree*, as if declining to answer were the most extreme disagreement on the scale. That is nonsense: "Don't know" is nonresponse wearing a number. It belongs with the `NA`s, but because it arrived coded as `5`, `na.rm = TRUE` sailed right past it. Our 1.14 is contaminated — only mildly here, since the 5s are 39 answers out of nearly 8,000, but wrong in a way that **no error message will ever flag**. `NA` was missingness R could see. The 5s are missingness in disguise. And it is not just this variable: ```{r count-others} voter |> count(pol_interest) ``` `pol_interest` runs 1–3 ("Not much" to "Very much") — and then 57 people at `4`, which the codebook reveals to be *Not sure*. Same trap, different costume. The other three issue items carry the 5 = Don't know code too, and two of them (`unimportant_budget`, `unimportant_abort`) have a further problem you may have noticed in their wording that we have not even touched yet. ::: {.callout-warning} ## Until further notice Do **not** report a mean, or any other summary, of the four issue items or of `pol_interest` from the raw data. Every such number is contaminated by disguised nonresponse, and the fix — telling R which codes are really missing, then dealing with what the recoding reveals — is precisely next week's lab. Until then, treat those five columns as radioactive: you may `count()` them, and nothing else. To be continued. ::: # Exercises Now you. Each exercise gives you scaffolded code with blanks (`___`); replace the blanks and run it. The chunks are marked not to run as-is — they will error until you fill them in. Fight with each one for a few minutes before opening the solution. ## Party identification by gender How does party identification break down by gender? Produce one table with a row for every combination. (Hint: `count()` accepts more than one variable.) ```{r ex1} #| eval: false voter |> count(___, ___) ``` In words, with the codebook at hand: which party's identifiers include more women than men? ::: {.callout-tip collapse="true"} ## Solution: party identification by gender ```{r ex1-sol} voter |> count(pid3, gender) ``` Decoding with the codebook (`pid3`: 1 Democrat, 2 Republican, 3 Other; `gender`: 1 Man, 2 Woman): among Democratic identifiers, women outnumber men (1,843 vs 1,796); among Republican identifiers it is the reverse (1,532 women vs 1,559 men). Note the two `NA` rows: the 89 people who skipped the party question still have a recorded gender, and `count()` refuses to sweep them under the rug. Get used to reading tables like this — a two-way breakdown of counts is the single most common object in survey research, and Week 8 gives it a formal treatment. ::: ## Post-graduates Two counts: (a) how many respondents hold a post-graduate degree? And (b) how many of them are women? Check the codebook for the right codes before you type. ```{r ex2} #| eval: false # (a) voter |> filter(___) |> dim() # (b) voter |> filter(___, ___) |> dim() ``` ::: {.callout-tip collapse="true"} ## Solution: post-graduates ```{r ex2-sol} # (a) post-grad is educ == 6 voter |> filter(educ == 6) |> dim() # (b) add the gender condition; woman is gender == 2 voter |> filter(educ == 6, gender == 2) |> dim() ``` 1,210 post-graduates, of whom 601 are women. The most common error here is `filter(educ = 6)` with a single equals sign — R's error message for that mistake is unusually helpful, so if you made it, read what it said. A `count()`-based route works too: `voter |> filter(educ == 6) |> count(gender)` gets (a) and (b) in one table. ::: ## The oldest respondent Who is the oldest respondent in the data — birth year, age in 2012, gender, and party? Sort the data so the oldest respondents come first and inspect the top of the result. (Hint: oldest means the *smallest* birth year.) ```{r ex3} #| eval: false voter |> arrange(___) |> select(birthyr, age, gender, pid3) |> head(3) ``` ::: {.callout-tip collapse="true"} ## Solution: the oldest respondent ```{r ex3-sol} voter |> arrange(birthyr) |> # ascending: smallest birth year first select(birthyr, age, gender, pid3) |> head(3) ``` The oldest respondents were born in 1921 — age 91 at the survey — and there are two of them, both women: one Democrat, one Republican. Note the plural: "who is the oldest" quietly assumed a unique answer, and the data declined to cooperate. Ties are why you inspect the top few rows rather than just row one. (If you wrote `arrange(desc(birthyr))`, you found the *youngest* respondents — born 1994, age 18. Sorting direction errors are silent, which makes them dangerous: always eyeball the result.) ::: ## Age in decades Create a variable `age_decade` giving the *decade of age* each respondent is in: 20 for ages 20–29, 30 for ages 30–39, and so on. Then tabulate it. The tool: integer division `%/%`, which divides and drops the remainder (`47 %/% 10` is `4`). ```{r ex4} #| eval: false voter |> mutate(age_decade = ___) |> count(___) ``` ::: {.callout-tip collapse="true"} ## Solution: age in decades ```{r ex4-sol} voter |> mutate(age_decade = 10 * (age %/% 10)) |> count(age_decade) ``` `age %/% 10` turns 47 into 4; multiplying by 10 turns it into the decade label 40. The table shows the survey's age profile at a glance — the fifties are the biggest decade (2,502 respondents) — and the top row is a small lesson: 13 respondents in "decade 10," who are simply the 18- and 19-year-olds. A perfectly correct computation produced a slightly odd-looking category, and only a human reading the output notices. We did not assign the result back to `voter` this time (no `voter <-`), so the column was printed and discarded; had we wanted to keep it, the assignment is one edit away. Coarsening a variable like this loses information — is that ever a good idea? Week 2 takes up exactly that question. ::: # Checklist By the end of this lab you should be able to: - [ ] Open Positron, write commands in a script, and run them with Ctrl+Enter / Cmd+Return — and say why scripts beat the console. - [ ] Explain the difference between `install.packages()` (once per machine) and `library()` (every session). - [ ] Work inside your course folder and load files with relative paths — and state the course rule about `setwd()` and absolute paths. - [ ] Load a CSV with `read_csv()` and inspect it with `dim()`, `names()`, `glimpse()`, and `head()`. - [ ] Explain why a dataset of bare integers is meaningless without its codebook, and make checking the codebook your reflex. - [ ] Use `select()`, `filter()`, `arrange()`, `mutate()`, and `count()`, chained with `|>`. - [ ] Derive a new variable (like `age`) with `mutate()` and sanity-check it. - [ ] Explain what `NA` is, what `na.rm = TRUE` does — and why it was *not* enough to make `mean(important_econ)` trustworthy. That last box is the one to sit with. You watched a clean-looking, plausible-sounding number get quietly poisoned by a code the software had no way to recognize as missing. Next week we do something about it.