Appendix B — Métodos Experimentales y Cuasi-experimentales

B.1 Motivación

  • Evidence-based programs

  • Statistical methods:

    • RCTs – promoted by agencies (e.g. J-PAL)
    • Quasi-experiments – look for situations “simulating” an experiment

  • Increased access to data and measurement of concepts

B.2 RCTs in practice

Freakonomics Radio

B.3 Some ideas

Objective: Measure the benefits (or lack of) that the program gives

  • Approach #1: Before versus after

  • Approach #2: Compare people with and without the program

B.4 Statistical concepts

Objective: Measure the benefits (or lack of) that the program gives

Ideal approach: Compare the same person with and without the program

  1. Potential Outcomes

  1. Treatment Effect

\[ TE=Y_i^T-Y_i^C \]

However… we can only observe “one” version of Sofia, not both!

Let’s imagine now that we compare health (Y) in two alternative worlds:

  • No one gets vaccination: everyone gets their \(Y_i^C\) level of health

  • Everyone gets vaccination: everyone gets their \(Y_i^T\) level of health

If we could look at both “versions” of the world we could calculate:

Average Treatment Effects: \(ATE=E[Y_i^T]-E[Y_i^C]\)

Now it looks impossible + extremely costly

But, what if we take a sample instead of looking at “everyone”

We can still get very close to \(E[Y_i^T]\) and \(E[Y_i^C]\) if we choose “wisely”

Problem with self-selection \(\Longrightarrow\) Bias

Standard errors. In this case, \(\bar{Y}^T-\bar{Y}^C\) gives us an “estimate” of \(ATE\). Thus we, need to create intervals in which \(ATE\) could be.

B.5 RCTs

Objective: Measure the benefits (or lack of) that the program gives

  • Solution: Randomization

Challenges found on RCTs:

  • Externalities. Mean that it is not possible to observe “control” in a pure form

Depending on the type of externality, we would be over- or under-estimating \(ATE\)

  • John Henry effects: a legendary American steel driver in the 1870s who, when he heard his output was being compared with that of a steam drill, worked so hard to outperform the machine that he died in the process.

  • Hawthrone effects: Hawthorne Works (Western Electric factory outside Chicago) commissioned a study to see if their workers would become more productive in higher or lower levels of light. The workers’ productivity seemed to improve when changes were made, and slumped when the study ended.

  • Attrition. Need to be careful with “selective attrition”

Could result from deaths, migration, unwillingness to continue participating

  • Partial participation: treatment is not always enforceable

\[ Y_1=\gamma_0+\gamma_1Z_i+U_i \]

\[ T_i=\eta_0+\eta_1Z_i+V_i \]

\(LATE\) (\(ATE\) on compliers)

\[ \dfrac{\partial Y}{\partial T}=\frac{\dfrac{\partial Y}{\partial Z}}{\dfrac{\partial T}{\partial Z}}=\frac{\gamma_1}{\eta_1} \]