Omega reliability affected by indicator direction?

[mattansb]

Not sure if this is the intended behavior:

library(lavaan)
library(semTools)

HS.model <- ' visual  =~ x1 + x2 + x3
              textual =~ x4 + x5 + x6
              speed   =~ x7 + x8 + x9 '

fit <- cfa(HS.model, data = HolzingerSwineford1939)

reliability(fit)
#>           visual   textual     speed
#> alpha  0.6261171 0.8827069 0.6884550
#> omega  0.6253180 0.8851754 0.6877600
#> omega2 0.6253180 0.8851754 0.6877600
#> omega3 0.6120052 0.8850608 0.6858417
#> avevar 0.3705589 0.7210163 0.4244883

HolzingerSwineford1939$x1 <- -1 * HolzingerSwineford1939$x1

HS.model <- ' visual  =~ x1 + x2 + x3
              textual =~ x4 + x5 + x6
              speed   =~ x7 + x8 + x9 '

fitN <- cfa(HS.model, data = HolzingerSwineford1939)

reliability(fitN)
#>             visual   textual     speed
#> alpha  -0.54666531 0.8827069 0.6884550
#> omega   0.02498404 0.8851754 0.6877600
#> omega2  0.02498404 0.8851754 0.6877600
#> omega3  0.02200703 0.8850608 0.6858417
#> avevar  0.37055894 0.7210163 0.4244883

^{Created on 2020-10-05 by the reprex package (v0.3.0)}

Shouldn't Omega "not care" about the sign of the loading coefficients?

[TDJorgensen]

Shouldn't Omega "not care" about the sign of the loading coefficients?

Only from the point of view that reliability is a proportion of variance. Notice that AVE is unaffected.

Alpha is an average covariance between pairs of items, which is why -1 * x1 turns it negative.

Omega is composite reliability. Would you make a composite by summing negatively and positively scored items? If you did, they would cancel each other out, which omega correctly reflects. Take the simple case of 2 items. The variance of the composite is the sum of the entire covariance matrix, so Var_1 + Var_2 + Cov_12 + Cov_21 (or simply Var_1 + Var_2 + 2Cov_21). When you negatively scored x1, you effectively imply calculating a composite by subtracting x1 from (x2 + x3), thereby removing a big chunk of common variance from the composite. Another way to think about it is that `-1 x1doesn't measurevisual` skills, but the lack thereof (or opposite), so of course including it in a composite would make it a noisy composite that doesn't reflect visual skills very well. Of course in practice, we always score our items in the same direction (e.g., reversing negatively worded items) before calculating a composite.

You can also compare the omega and AVE formulas (see p. 153) to see why the former changes with direction. Omega sums loadings before squaring, whereas AVE squares loadings before summing. AVE is the average R^2 across items. Omega is also about the proportion of variance, but not of items, rather of the composite, so the direction of loadings (and by implication, of how the items are scored) should by definition impact the interpretation of a composite and its reliability.

[mattansb]

Oh, I see - thanks, that clears things up!