search

sem-in-r / seminr

Natural feeling domain-specific language for building structural equation models in R for estimation by covariance-based methods (like LISREL/Lavaan) or partial least squares (like SmartPLS)
58 stars 19 forks source link

Sign Indeterminancy during Bootstrapping #163

Open kleinjulius opened 4 years ago

kleinjulius commented 4 years ago

Hi all,

Thank you for your awesome work on this R Package!

I am running into an issue when trying to assess my PLS model with Bootstrapping. During the bootstrapping analysis, I am obtaining rather wide confidence intervals, indicating that most of the path coefficients and many indicator loadings and weights are statistically non-significant. I am wondering if this may be due to sign indeterminacy. It would be great if it was possible to use sign-change options (individual or construct) in SEMinR. Any comments or guidance on how to deal with this issue would be greatly appreciated.

Thank you in advance

soumyaray commented 4 years ago

Thanks for the issue submission @kleinjulius !

Are the issues you are facing around a single problematic construct or is it widespread throughout the model?

We will start right away to see about implementing sign-change options.

In the meantime, I might suggest some diagnostic code to help you visualize what kinds of estimates are coming out of the bootstrap. First, note that for a single path parameter (say, between constructs "Image" and "Expectation"), all 5000 estimates during bootstrapping are available using:

boot_mobi_pls$boot_paths["Image", "Expectation",]

You can visualize these using a histogram:

hist(boot_mobi_pls$boot_paths["Image","Expectation",])

You might want to try this out to see if you are getting any strange distributions for path parameters – say, a bimodal distribution.

kleinjulius commented 4 years ago

Hi @soumyaray,

thank you for your hint. The issue occurs across different constructs but varies in its severity. Below I have attached three patterns which occur:

The first one is pretty much in line with what i expected: The path coefficient is negative, with the bootstrap CI containing zero.

Boot_Path-TZDist-Performance

The second and third patterns are more concerning and lead me to suspect that sign indeterminacy might be an issue:

BootPath_Engagement-Performance Boot_Loading_TempDisp3-Dispersion

soumyaray commented 3 years ago

Hi @kleinjulius I'd like to try out some sign change strategies on your dataset. Kindly email me at: soumya [dot] ray [at] gmail [dot] com

I'd like to discuss this with you in more detail.