Multilevel modeling and ordinary least squares regression: How comparable are they?

Studies analyzing clustered data sets using both multilevel models (MLMs) and ordinary least squares (OLS) regression have generally concluded that resulting point estimates, but not the standard errors, are comparable with each other. However, the accuracy of the estimates of OLS models is important to consider, as several alternative techniques (e.g., bootstrapping) used when analyzing clustered data sets only make adjustments to standard errors but not to the regression coefficients. Using a Monte Carlo simulation, we analyzed 54,000 data sets using both MLM and OLS under varying conditions and we show that coefficients of not just OLS models, but MLMs as well, may be biased when relevant higher-level variables are omitted from a model, a situation that is likely to occur when using large-scale, secondary data sets. However, we demonstrate that by including aggregated level-one variables at the higher level, the resulting bias can be effectively removed.