So I don’t listen to the Quantitude podcast. It’s really difficult for me to get into it when my playlist includes podcasts with episodes titles like: Interstellar Jihad. However, I saw several tweets inspired by the SEM vs. regression episode. And I have too many thoughts, opinions, feelings to stay quiet. I lay them out here.
SEM is regression
So this is pretty much established and known by methodologists. The basic SEM models are mathematically identical to very specific regression models e.g.
- A multi-dimensional CFA is a multilevel regression (with the long-format item response data) where each random effect (by respondent) is a factor, the loadings are random effect SDs that vary by item (non-linear regression), varying error variances pertain to heteroskedasticity, means are item intercepts (often estimated as fixed effects).
- A Full SEM (measurement + structural) \(\equiv\) some type of CFA \(=\) multilevel regression.
- Binary SEM (using tetrachoric correlations) \(=\) multilevel probit regression
- Ordinal SEM (using polychoric correlations) \(=\) multilevel cumulative-probit regression
A case of cultural differences
Now differences exist between the practice of SEM and standard regression approaches. If your SEM introduction was anything like mine, you learned that:
- measurement error biases the estimation of structural paths within a system of equations and SEM mitigates against this bias
- the LISREL model (whether 4 or 5 or 9 or 20 matrices version) is at the core of SEM - more on this later
- After estimating a model that matches your theory, you need to check model-data fit often using global fit indices (I don’t like this) - a lot of SEM energy is expended on this last part
In practice, SEMers have developed a variety of software to enable estimating parameters in a system of equations. While the most common regression methods and software focus on univariate outcomes. Interestingly, you could fit the exact same model under both approaches and the gold-standard model validation techniques would be completely different. Hence my comment that the differences are cultural.
The actual difference is LISREL
From a methodological perspective, the actual difference is LISREL. The LISREL equations are the solution to a very specific formulation of a regression problem that allow us compute all the parameters we like to interpret in an SEM.
So SEM estimation works like this for the most part:
- Compute covariance matrix of data
- Solve LISREL equations
Are the data binary/ordinal? Instead, calculate a tetrachoric/polychoric correlation matrix, do LISREL.
And LISREL works nicely for the very precise regression problem it’s set up to solve. With Bayesian estimation, your models run so much faster when using the LISREL equations.
But too much LISREL, not enough measurement error
If you were in an SEM class, you might believe that SEM is the way to address measurement error problems. Some teachers might mention adjustments for attenuation.
You would likely miss out on the rich discussions of measurement error developed in the wider regression literature e.g. a pretty good review of measurement error in econometrics by Hausman, book on different approaches to measurement error in regression.
Many folks I interact with tend to believe that regression approaches often ignore measurement error and SEMs do not. I’d counter that SEM presents just one solution for dealing with measurement.
LISREL works well for normal-data normal-model no manifest covariates models
Given the focus on measurement error, the LISREL model is only ideal in the normal-data normal-model situation no manifest covariates. Why so?
In the regression problem LISREL is set up to solve, the indicators form the outcome variable. Something I’ve never heard talked about in an SEM class is how to deal with measurement error in these variables themselves.
That may be because it is already known that measurement error in linear regression outcomes does not bias estimates, however, there is some loss of power. Hence, continuous indicators with measurement error themselves (likely a given in practice) will result in unbiased estimates – see Hausman paper above.
Covariance matrix \(\to\) LISREL likely fails with discrete indicators
However, measurement error in binary indicators actually leads to biased parameter estimates. Measurement error manifests as miss-classification – a yes is recorded as no and vice-versa. Low rates of miss-classification can lead to significant bias in estimates. And in my experience, SEM research is often nitpicky over less consequential issues.
Hence, the standard tetrachoric correlation \(\to\) LISREL pipeline fails to address the measurement error problems one set out to solve.
It actually happens that one approach to this problem is the 4PL IRT model which can be motivated as a measurement error model for binary indicators with miss-classification. What IRT folks call “guessing” is simply a wrong response classified as a correct one, while “slipping” is a correct response classified as a wrong one i.e. measurement error.
However, if you practiced SEM, you might believe that computing tetrachoric correlations is the gold-standard with your major problem being: how trustworthy is your approach to computing tetrachoric correlations?
But the tetrachoric coefficients are already incorrect in the presence of measurement error. LISREL won’t save you. This problem exists for other kinds of discrete data, count, ordinal, …
LISREL likely fails for SEMs with observed covariates (e.g. MIMIC model)
Another case is the multiple indicator multiple causes (MIMIC) model where folks often have covariates or “causes”. Unlike measurement error in indicators, measurement error in covariates will lead to biased estimates. And given that the whole system of variables is computed together, measurement error in covariates can easily propagate across the system affecting all other aspects of the model. This likely happens in practice.
But again, because of focus on LISREL, this is not focused on much or it is assumed away. The goal is to get an acceptable covariance matrix then do LISREL.
In closing, …
I have no closing thoughts. My goal here is to suggest that even for things standard SEMs are supposedly good at, they struggle. And as a methods person, I spend time reading the broader regression literature to do my SEM better.
I also have a high degree of skepticism towards SEM applications. I am not convinced that the ability to test a system of equations in the way SEM software afford is a good thing. I believe SEM results often provide unwarranted confidence about complex patterns in data especially when model-data fit is deemed acceptable.
A regression though much less ambitious on the surface is often small enough to do different kinds of elaborate checks that can themselves become unwieldy.
However, if you stay humble in your beliefs about patterns in data, SEM cultural practices can be a very interesting exploratory tool. Also, if you’re serious about handling measurement error in your research, SEM is not enough.
I think I should note a difficulty in discussing this topic. I believe SEM \(=\) regression, so what’s the point of this post? But then there are folks who believe SEM \(=\) LISREL. I think in this day and age, that view can be a distraction and sometimes even detrimental …
Comments powered by Talkyard.