9.1 Internal and External Validity
Key Concept 9.1
Internal and External Validity
A statistical analysis has internal validity if the statistical inference made about causal effects are valid for the considered population.
An analysis is said to have external validity if inferences and conclusion are valid for the studies’ population and can be generalized to other populations and settings.
Threats to Internal Validity
There are two conditions for internal validity to exist:
The estimator of the causal effect, which is measured the coefficient(s) of interest, should be unbiased and consistent.
Statistical inference is valid, that is, hypothesis tests should have the desired size and confidence intervals should have the desired coverage probability.
In multiple regression, we estimate the model coefficients using OLS. Thus for condition 1. to be fulfilled we need the OLS estimator to be unbiased and consistent. For the second condition to be valid, the standard errors must be valid such that hypothesis testing and computation of confidence intervals yield results that are trustworthy. Remember that a sufficient condition for conditions 1. and 2. to be fulfilled is that the assumptions of Key Concept 6.4 hold.
Threats to External Validity
External validity might be invalid
if there are differences between the population studied and the population of interest.
if there are differences in the settings of the considered populations, e.g., the legal framework or the time of the investigation.