This book is in Open Review. We want your feedback to make the book better for you and other students. You may annotate some text by selecting it with the cursor and then click the on the pop-up menu. You can also see the annotations of others: click the in the upper right hand corner of the page

13 Experiments and Quasi-Experiments

This chapter discusses statistical tools that are commonly applied in program evaluation, where interest lies in measuring the causal effects of programs, policies or other interventions. An optimal research design for this purpose is what statisticians call an ideal randomized controlled experiment. The basic idea is to randomly assign subjects to two different groups, one that receives the treatment (the treatment group) and one that does not (the control group) and to compare outcomes for both groups in order to get an estimate of the average treatment effect.

Such experimental data is fundamentally different from observational data. For example, one might use a randomized controlled experiment to measure how much the performance of students in a standardized test differs between two classes where one has a “regular”" student-teacher ratio and the other one has fewer students. The data produced by such an experiment would be different from, e.g., the observed cross-section data on the students’ performance used throughout Chapters 4 to 8 where class sizes are not randomly assigned to students but instead are the results of an economic decision where educational objectives and budgetary aspects were balanced.

For economists, randomized controlled experiments are often difficult or even indefeasible to implement. For example, due to ethic, moral and legal reasons it is practically impossible for a business owner to estimate the causal effect on the productivity of workers of setting them under psychological stress using an experiment where workers are randomly assigned either to the treatment group that is under time pressure or to the control group where work is under regular conditions, at best without knowledge of being in an experiment (see the box The Hawthorne Effect on p. 528 of the book).

However, sometimes external circumstances produce what is called a quasi-experiment or natural experiment. This “as if” randomness allows for estimation of causal effects that are of interest for economists using tools which are very similar to those valid for ideal randomized controlled experiments. These tools draw heavily on the theory of multiple regression and also on IV regression (see Chapter 12). We will review the core aspects of these methods and demonstrate how to apply them in R using the STAR data set (see the description of the data set).

The following packages and their dependencies are needed for reproduction of the code chunks presented throughout this chapter:

  • AER (Kleiber & Zeileis, 2017)
  • dplyr (Wickham et al., 2018)
  • MASS (Ripley, 2018)
  • mvtnorm (Genz et al., 2018)
  • rddtools (Stigler & Quast, 2015)
  • scales (Wickham, 2018)
  • stargazer(Hlavac, 2018)
  • tidyr (Wickham & Henry, 2018)

Make sure the following code chunk runs without any errors.

library(AER)
library(dplyr)
library(MASS)
library(mvtnorm)
library(rddtools)
library(scales)
library(stargazer)
library(tidyr)

References

Kleiber, C., & Zeileis, A. (2017). AER: Applied Econometrics with R (Version 1.2-5). Retrieved from https://CRAN.R-project.org/package=AER

Wickham, H., François, R., Henry, L., & Müller, K. (2018). dplyr: A Grammar of Data Manipulation (Version 0.7.8). Retrieved from https://CRAN.R-project.org/package=dplyr

Ripley, B. (2018). MASS: Support Functions and Datasets for Venables and Ripley’s MASS (Version 7.3-51.1). Retrieved from https://CRAN.R-project.org/package=MASS

Genz, A., Bretz, F., Miwa, T., Mi, X., & Hothorn, T. (2018). mvtnorm: Multivariate Normal and t Distributions (Version 1.0-8). Retrieved from https://CRAN.R-project.org/package=mvtnorm

Stigler, M., & Quast, B. (2015). rddtools: Toolbox for Regression Discontinuity Design (’RDD’) (Version 0.4.0). Retrieved from https://CRAN.R-project.org/package=rddtools

Wickham, H. (2018). scales: Scale Functions for Visualization (Version 1.0.0). Retrieved from https://CRAN.R-project.org/package=scales

Hlavac, M. (2018). stargazer: Well-Formatted Regression and Summary Statistics Tables (Version 5.2.2). Retrieved from https://CRAN.R-project.org/package=stargazer

Wickham, H., & Henry, L. (2018). tidyr: Easily Tidy Data with ’spread()’ and ’gather()’ Functions (Version 0.8.2). Retrieved from https://CRAN.R-project.org/package=tidyr