Cowell Cressida

Testing non-nested models without losing your mind

If you've ever tried to compare two regression specifications where neither one contains the other, you already know the usual t-test and F-test won't cut it. That's where the Cowell Cressida procedure comes in. It's essentially a formalized way of running the kind of mental juggling act you'd do anyway when your model keeps getting rejected by the data. The core idea is straightforward enough. You take model A and model B, estimate both on the same dataset, then construct a joint model that includes the key regressors from each side. Once you have that combined specification, you run a set of restrictions and let the test statistic tell you whether one model systematically outperforms the other or whether both are just equally bad at describing the variation in your data.

cowell cressida in practice

Here's the step-by-step, stripped of the textbook padding: Step one is estimating your two competing models separately. Call them M1 and M2. Write down the fitted values from each. Don't skip this — you need those predictions later.

Step two is creating the augmented model. Take the original regressors from M1, add the original regressors from M2, and run OLS on the union of both sets. If either model has generated regressors like lagged dependent variables or instrumental variables, make sure you carry those over too. I once forgot a lagged dependent variable when constructing the augmentation and got a test statistic that was completely wrong because the error structure had changed silently. Took me three hours to notice. The fix was just re-estimating with the full lag structure included. Step three involves the actual test statistic. You're looking at a Wald-type or likelihood-ratio-style comparison between the restricted version (just M1 or just M2) and the unrestricted augmented model. Under the null that one model is correctly specified, the statistic should follow a chi-squared distribution with degrees of freedom equal to the number of additional parameters you introduced. In most econometrics packages you can run this in about five minutes once the models are estimated.

Step four is interpretation. If the statistic is significant, you reject the null. That means the model you were testing against is missing something that the other model captures. If it's not significant, both models are doing roughly the same job and you pick based on parsimony or theoretical grounds instead of the data alone. One thing beginners consistently miss: the test only works cleanly when both models are estimated on exactly the same observations. If M1 drops missing values in a different pattern than M2, your augmented model will silently use a third sample and invalidate everything. Always check the observation count across all three estimations. I use a simple tabulation command before running the test to confirm alignment.

There are also edge cases where this approach breaks down. When your two models contain heavily collinear regressors, the augmented specification can become numerically unstable and the test statistic inflates artificially. I've seen this happen with macroeconomic time series where both models ended up using slightly different transformations of the same underlying variables. The workaround is to check variance inflation factors in the augmented model before trusting the result. If VIFs exceed 10 or 15, the test isn't reliable and you should fall back to informational criteria like AIC or BIC computed on each model separately. Another limitation worth noting: the Cowell Cressida framework assumes your errors are well-behaved. Heteroskedasticity or autocorrelation will bias the standard errors and therefore the test statistic. Robust standard errors help, but they don't fully solve the problem when the misspecification is severe. In those cases a pre-test for heteroskedasticity followed by HAC corrections is the minimum acceptable approach.

The procedure itself is available through most standard econometrics software. In R, you can reconstruct it manually after estimating both models with lm() or glm(). Stack the regressors, run the augmented fit, then use linearHypothesis() from the car package for the Wald test. In Stata, after estimating each model with regress, you can create the pooled specification and run test on the added terms. Some packages also offer automated non-nested testing functions, but they usually wrap the same logic with less transparency about what's actually happening under the hood. The take-away is that Cowell Cressida gives you a structured answer to a question that otherwise feels subjective. It doesn't guarantee you'll pick the right model, but it does force you to confront the difference between your two specifications with a number instead of an intuition. Just don't treat a non-significant result as proof that both models are correct. It only means you haven't found evidence that one is clearly better than the other given your data and sample size.