[Bug]: Problem with using ML estimator and FIML for multi-group SEM

[amielaugustin]

JASP Version

0.18.2

Commit ID

No response

JASP Module

SEM

What analysis are you seeing the problem on?

Multigroup (3 groups) SEM

What OS are you seeing the problem on?

Windows 10

Bug Description

Hi everyone,

I'm not sure if it's more a bug report than a feature request. I'm trying to run the analysis from an article (so run the same analysis) which implies a set of ordered data and use a multigroup SEM. I think that my questions are quite simple : is it impossible to use ML estimator and FIML for missing datas when we have ordered data ? I have a lot of items with scale responses like the answers are "Totally disagree, .. , Totally agree".

Because for your information, I receive this kind of error display when I try : " FIML is not supported for categorical data." or "Error in Model 1 - Estimation failed message: lavaan ERROR: estimator ML for ordered data is not supported yet, use WLSMV instead."

The problem is that in the analysis that I need to replicate, they use these estimators, but with another software than JASP. I saw somewhere that not all of software can use these estimators for ordered/categorical data.

What do you think about this ?

Thanks a lot by advance for you attention.

Augustin

Expected Behaviour

I wanted to run the analysis with ML estimator and FIML with my ordered data set.

Steps to Reproduce

1. I selected ML Estimator in the "Estimation options" section
2. I also selected FIML for missing data handling
3. Both selections gave me the error display that I've described above ...

Log (if any)

No response

Final Checklist

• [ ] I have included a screenshot showcasing the issue, if possible.
• [ ] I have included a JASP file (zipped) or data file that causes the crash/bug, if applicable.
• [X] I have accurately described the bug, and steps to reproduce it.

[github-actions[bot]]

@amielaugustin, thanks for taking the time to create this issue. If possible (and applicable), please upload to the issue website (https://github.com/jasp-stats/jasp-issues/issues/2476) a screenshot showcasing the problem, and/or a compressed (zipped) .jasp file or the data file that causes the issue. If you would prefer not to make your data publicly available, you can send your file(s) directly to us, issues@jasp-stats.org

[amielaugustin]

![Uploading Capture d’écran (24).png…]()

[patc3]

Ordered data doesn't use ML estimation

As we discussed in this forum post, probably that the authors you're replicating did not in fact treat their variables as ordered but as scale instead (as most software packages do by default). In any case, the SEM library used in JASP, lavaan, doesn't do ML/FIML when the data are treated as ordered.

[amielaugustin]

Dear Patrick,

Thanks for answering. Following your advice, I switched the data on a continuous scale. Unfortunately, I have still some troubles with my model. Now, I have to face the following error message : " Estimation failed message : lavaan ERROR: invalid value(s) in test= argument: ". I tried to use the lavaan syntax as explained in https://lavaan.ugent.be/ and in an example video with industrialization and democracy in 60's and 65's. I think I missed something; I am not sure of my model lavaan syntax because I've more latent variables and mediation paths than in the example. Can I send you some screen shots of my code and the related article ? If you have time to have a look, thank you very much because it helps me a lot, and if you don't have time, let me know and I'll post it on the forum.

The link for the article is : https://owners-ercproject.eu/wp-content/uploads/2022/05/Storz-Bilali-et-al.-2021.pdf

It's Study 2.

I didn't write anything in the "release constraints" box for the moment (although I have to because they freed some paths). Because when I do it I have another error message.. I tried something like : inclusive victimhood ~ c(a,a,NA)*shared ownership, and I received this error message : Error in check_lengths(string, pattern, replacement): argument “replacement” is missing, with no default

tryCatch(suppressWarnings({returnVal <- eval(parse(text = .rCode))}), error = function(e) {.setRError(paste0(toString(e), “\n”,paste0(sys.calls(), collapse = “\n”))) })

tryCatchList(expr, classes, parentenv, handlers)

tryCatchOne(expr, names, parentenv, handlers[[1]]) value[3].

If you need more information, feel free to ask.

Thanks a lot for reading me, it helps me a lot.

Augustin

---

De : Patrick Coulombe @.> Envoyé : dimanche 17 décembre 2023 18:53 À : jasp-stats/jasp-issues @.> Cc : amielaugustin @.>; Mention @.> Objet : Re: [jasp-stats/jasp-issues] [Bug]: Problem with using ML estimator and FIML for multi-group SEM (Issue #2476)

Ordered data doesn't use ML estimation

As we discussed in this forum posthttps://forum.cogsci.nl/discussion/9032/problem-with-using-ml-estimator-and-fiml, probably that the authors you're replicating did not in fact treat their variables as ordered but as scale instead (as most software packages do by default). In any case, the SEM library used in JASP, lavaan, doesn't do ML/FIML when the data are treated as ordered.

— Reply to this email directly, view it on GitHubhttps://github.com/jasp-stats/jasp-issues/issues/2476#issuecomment-1859235868, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BDPCLLZ6NEKZ3Y3WC4XQE43YJ4WRXAVCNFSM6AAAAABAWS6SH2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNJZGIZTKOBWHA. You are receiving this because you were mentioned.

[patc3]

Hi Augustin, before I dig further, did you select a variable in the multi group option? Also, have you tried setting the third label to something other than NA (e.g. b)?

Also could you copy paste your code here please?

[amielaugustin]

Hi Patrick,

Yes, I selected a variable in the multi-group option. I've just tried now to use another label (b) instead of NA and it gives me the same error message.

---

De : Patrick Coulombe @.> Envoyé : mercredi 20 décembre 2023 16:05 À : jasp-stats/jasp-issues @.> Cc : amielaugustin @.>; Mention @.> Objet : Re: [jasp-stats/jasp-issues] [Bug]: Problem with using ML estimator and FIML for multi-group SEM (Issue #2476)

Hi Augustin, before I dig further, did you select a variable in the multi group option? Also, have you tried setting the third label to something other than NA (e.g. b)?

— Reply to this email directly, view it on GitHubhttps://github.com/jasp-stats/jasp-issues/issues/2476#issuecomment-1864631177, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BDPCLLYJONJQUBY5ZPLQFCTYKL5EZAVCNFSM6AAAAABAWS6SH2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNRUGYZTCMJXG4. You are receiving this because you were mentioned.

[patc3]

Can you please post exactly what you're doing to get your error messages. What analysis, what's your lavaan syntax, what inputs (eg checkboxes) are you using, etc. Give enough info that I could reproduce your analysis exactly on my computer... Help me help you, in other words

[patc3]

There's a tutorial in French here in case it helps (written tutorial + YouTube video) https://allostatsesg.uqam.ca/modelisation-par-equations-structurelles-sem-avec-jasp/

(No multigroup unfortunately)

[amielaugustin]

I opened the data set in excel which is included on the top of the file " Study 2 " from the section " data availability statement (OSF) " at the end of the article (p.13). Then I clicked on "Enable editing" on Excel, and I saved the file in a .csv (comma delimited) type. I opened JASP and then opened the .csv file. Then I put all the variable on a continuous scale by clicking on the type of variable like you already told me, except the gender, eth1_new (the grouping variable). Then I clicked on the SEM analysis button and I chose "Structural Equation Modeling". Then I enter my code in the lavaan syntax box which I write as follow :

' # latent variables

inclusive victimhood =˜ vc_1 + vc_2

competitive victimhood =˜ vc_3 + vc_4

shared ownership =˜ cpob_1_1 + cpob_1_2 + cpob_1_3

joint political decision-making =˜ shr_1 + shr_2 + shr_3

regressions

shared ownership ˜ inclusive victimhood + competitive victimhood

joint political decision-making ˜ shared ownership + inclusive victimhood + competitive victimhood '

Then I changed the " Estimation options " box by clicking on Boostrap Error calculation with a 10000 Bootstrap sample (I found this number on the code of the author in one file of the OSF : 3b SEM - no controls - unconstrained - Table S2.2). Then I selected the ML estimator and FIML missing data handling. I also chose Mplus Emulation because it seems they did it. In the Multigroup SEM box, I chose the grouping variable "eth1_new" and I selected two equality constraints : the loadings because after an invariance measurement, they chose a metric invariance, and the regressions (which is the regression coefficients ?) because the author said that she freed some paths (two with the most divergent coefficients between groups) of a constrained model to have a partially constrained model which have a better fit than the constrained model and a fit as good as the unconstrained one. So in the release constraints box I tried to apply the lavaan syntax by writing " shared ownership ~ c(a,a,NA)*inclusive victimhood

joint political decision making ~ c(a,a,NA)*inclusive victimhood" because the third group is the Serb population in Kosovo, and the one that must be freed.

For your information, you have the figure of the SEM applied and chosen by the author in the Study 2, which could help you to understand the link between the variable. I didn't tap a line for the link between inclusive victimhood and competitive victimhood because I think it was not specified in the model first, it was not related to a hypothesis.

I don't know why but by repeating all the process from the beginning now it works the model is running, except the release box which is still not working. So I have fixed the problem of error messages (except for the release box) but I'm not sure I write the good syntax.. I don't know if I have to write more lines, I mean I don't know if this syntax describes well the direct and indirect effects between all the latent variables.

Here is the link for the article, which would be very useful if you want to try to reproduce what I've done (and find the data in the OSF p.13) : https://owners-ercproject.eu/wp-content/uploads/2022/05/Storz-Bilali-et-al.-2021.pdf

I joined you my results file.

For the release box, here is the error message :

' Error in check_lengths(string, pattern, replacement): argument “replacement” is missing, with no default

tryCatch(suppressWarnings({returnVal <- eval(parse(text = .rCode))}), error = function(e) {.setRError(paste0(toString(e), “\n”,paste0(sys.calls(), collapse = “\n”))) })

tryCatchList(expr, classes, parentenv, handlers)

tryCatchOne(expr, names, parentenv, handlers[[1]]) value[3] '

Thanks a lot for your consideration and advice.

Augustin

---

De : Patrick Coulombe @.> Envoyé : mercredi 20 décembre 2023 16:26 À : jasp-stats/jasp-issues @.> Cc : amielaugustin @.>; Mention @.> Objet : Re: [jasp-stats/jasp-issues] [Bug]: Problem with using ML estimator and FIML for multi-group SEM (Issue #2476)

Can you please post exactly what you're doing to get your error messages. What analysis, what's your lavaan syntax, what inputs (eg checkboxes) are you using, etc. Give enough info that I could reproduce your analysis exactly on my computer... Help me help you, in other words

— Reply to this email directly, view it on GitHubhttps://github.com/jasp-stats/jasp-issues/issues/2476#issuecomment-1864668188, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BDPCLL74UMNJAEWSCR5IUOTYKL7RJAVCNFSM6AAAAABAWS6SH2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNRUGY3DQMJYHA. You are receiving this because you were mentioned.

[patc3]

Hi Augustin, I also couldn't get release constraints to work. The way release constraints is supposed to work, as far as I know, you simply write the formula you want to free across groups, e.g. sharedownership ~ inclusivevictimhood. Also one thing's for sure is you shouldn't use NA as a label: same labels get the same values, and different labels get different values, so instead of doing c(a,a,NA), you could simply do c(a,a,b)

That said, the easiest (seeing as the release constraints really isn't working easily) would be to do it directly in your model syntax. So for example:

# regressions
sharedownership ~ c(a,a,b)*inclusivevictimhood + competitivevictimhood

[patc3]

Your model seems to be correctly specified.

Here's what I have (with the released constraints):

inclusivevictimhood =~ vc_1 + vc_2
competitivevictimhood =~ vc_3 + vc_4
#inclusivevictimhood ~~ competitivevictimhood

sharedownership =~ cpob_1_1 + cpob_1_2 + cpob_1_3
sharedownership ~ c(ainc1,ainc1,ainc2)*inclusivevictimhood + c(acomp,acomp,acomp)*competitivevictimhood

supportpolitical =~ shr_1 + shr_2 + shr_3
supportpolitical ~ c(b,b,b)*sharedownership + c(c1,c1,c2)*inclusivevictimhood + competitivevictimhood

# indirect effects
ind_through_inclusive1 := ainc1*b
ind_through_inclusive_kosovo := ainc2*b
ind_through_competitive := acomp*b

To calculate indirect effects, you need to create new parameters using the := lavaan operator. An indirect effect is the product of serial paths, so in this case from victimhood to ownership to political support.

[amielaugustin]

Thanks a lot Patrick. I'll try this afternoon and let you know.

Best, Augustin

Envoyé à partir de Outlook pour Androidhttps://aka.ms/AAb9ysg

---

From: Patrick Coulombe @.> Sent: Thursday, December 21, 2023 11:32:25 PM To: jasp-stats/jasp-issues @.> Cc: amielaugustin @.>; Mention @.> Subject: Re: [jasp-stats/jasp-issues] [Bug]: Problem with using ML estimator and FIML for multi-group SEM (Issue #2476)

Your model seems to be correctly specified.

Here's what I have (with the released constraints):

inclusivevictimhood =~ vc_1 + vc_2 competitivevictimhood =~ vc_3 + vc_4

inclusivevictimhood ~~ competitivevictimhood

sharedownership =~ cpob_1_1 + cpob_1_2 + cpob_1_3 sharedownership ~ c(ainc1,ainc1,ainc2)inclusivevictimhood + c(acomp,acomp,acomp)competitivevictimhood

supportpolitical =~ shr_1 + shr_2 + shr_3 supportpolitical ~ c(b,b,b)sharedownership + c(c1,c1,c2)inclusivevictimhood + competitivevictimhood

indirect effects

ind_through_inclusive1 := ainc1b ind_through_inclusive_kosovo := ainc2b ind_through_competitive := acomp*b

To calculate indirect effects, you need to create new parameters using the := lavaan operator. An indirect effect is the product of serial paths, so in this case from victimhood to ownership to political support.

— Reply to this email directly, view it on GitHubhttps://github.com/jasp-stats/jasp-issues/issues/2476#issuecomment-1867012362, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BDPCLLYDPXIA7JSEXDJUVZLYKS2HTAVCNFSM6AAAAABAWS6SH2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNRXGAYTEMZWGI. You are receiving this because you were mentioned.Message ID: @.***>

[tomtomme]

@amielaugustin Do you have any updates on this? Also your jasp file did not upload. Please rename it to zip and then drag and drop it here. Email does not work for uploading files.

[amielaugustin]

Hello tomtomme,

Sorry for the delay, I haven't seen your message before. Yes the problem is solved, I just needed to treat my data as continuous.

Best, Augustin

[tomtomme]

Thx for the info :) Good to know