Evaluating the Econometric Evaluations of Training Programs with Experimental Data

Evaluating the Econometric Evaluations of Training Programs with Experimental Data

Authors: Robert J. LaLonde

Citation: LaLonde, Robert J. (1986). Evaluating the Econometric Evaluations of Training Programs with Experimental Data. The American Economic Review, 76(4), 604--620.

Abstract: This paper compares the effect on trainee earnings of an employment program that was run as a field experiment where participants were randomly assigned to treatment and control groups with the estimates that would have been produced by an econometrician. This comparison shows that many of the econometric procedures do not replicate the experimentally determined results, and it suggests that researchers should be aware of the potential for specification errors in other nonexperimental evaluations.

Reading Notes

Objective

To compare experimental and nonexperimental estimates of the effect of training programs on wages

Background

National Supported Work Demonstration (NSW) assigned qualified applicants to training programs randomly.

There were some differences in jobs conducted between participants: women tended to be assigned service jobs, men construction jobs

Data & Key Variables

National Supported Work program data

Control group from CPS and PSID

Methodology

For experimental estimates: difference in differences

Non-experimental estimates using a variety of methods: OLS with a dummy for participation, fixed effects, diff-in-diff and a two-step method to account for any correlation between participation and observables

Results

Using the experimental data - female earnings increased by $851 and male earnings increased by $886

Different specifications on nonexperimental estimates give vastly different results.

Specification tests would lead researchers to vastly overestimate effect of training programs

Key Table/Figure