Dummy variables in R

Dummy variables are important but also cause much frustration in intro-stat courses. Below I will demonstrate the concept via a linear regression model.

The basic idea is that a factor $f$ with $k$ levels can be replaced by $k-1$ dummy variables that act as switches to select different levels. When all switches are turned off, the reference level is chosen. Mathematically, let $f$ be the factor with levels $l_0,l_1,\dots ,l_{k-1}$, i.e. $f\in \{l_0,l_1,\dots ,l_{k-1}\}$. By convention, let $l_0$ be the reference level chosen by the user. Now introduce the $k-1$ dummy variables $z_1,z_2,\dots ,z_{k-1}$ defined by $$z_i=\{\begin{array}{cc}1& \text{if}f=l_i\\ 0& \text{otherwise}\end{array}$$ for $i\in \{1,2,\dots ,k-1\}$. Note that

Assume that we are interested in the ANOVA model given by the R formula y ~ f (e.g. lm(y ~ f)). Then R automatically translates this into the model $$y={\beta }_0+{\beta }_1{z}_1+{\beta }_2{z}_2+\epsilon$$ with dummy variables $z_1$ and $z_2$ as defined above. This can be illustrated in R as follows:

f <- factor(c("l0", "l1", "l2"))
as.data.frame(model.matrix(~ f))
##   (Intercept) fl1 fl2
## 1           1   0   0
## 2           1   1   0
## 3           1   0   1

So the first row is $f=l_0$, the second $f=l_1$, and the third $f=l_2$.

Here we see that the intercept is the constant (“silent”) $1$ in front of ${\beta }_0$ such that ${\beta }_0$ is always included. The parameter ${\beta }_0$ is the mean of the $y$’s for $f=l_0$. Notice the column name fl1; this refers to the difference in mean of $y$ between $f=l_0$ and $f=l_1$. This can be seen by inspecting row two above. The convention in R is to concatenate the factor (variable) name, here f, with the level, here l1.

As seen, the first level was taken as the reference level (silently by R). This is the convention: the first level of the factor is the reference level:

f <- factor(c("l0", "l1", "l2"), level = c("l1", "l0", "l2"))
as.data.frame(model.matrix(~ f))
##   (Intercept) fl0 fl2
## 1           1   1   0
## 2           1   0   0
## 3           1   0   1

Sometimes the relevel() function is useful:

f <- factor(c("l0", "l1", "l2"))
f <- relevel(f, ref = "l2") # ref: the reference level
as.data.frame(model.matrix(~ f))
##   (Intercept) fl0 fl1
## 1           1   1   0
## 2           1   0   1
## 3           1   0   0