search

jslefche / piecewiseSEM

Piecewise Structural Equation Modeling in R
157 stars 49 forks source link

No independence claims present. Tests of directed separation not possible. #271

Open JamieMcDevittIrwin opened 1 year ago

JamieMcDevittIrwin commented 1 year ago

Hello,

I'm trying to use piecewiseSEM and am running into an error that I can't find a solution for online anywhere. Whenever I look at the summary() of my psem model, it tells me "No independence claims present. Tests of directed separation not possible” under "Tests of directed separation:", "Coefficients:", and "Individual R-squared:" in the output.

My data looks like this:

str(sem.df) 'data.frame': 213 obs. of 7 variables: $ Herbivores : num 40.7 40.7 40.7 40.7 40.7 ... $ Shark_Sum_Abundance: num 3.23 3.23 3.23 3.23 3.23 ... $ Mesopredators : num 1.67 1.67 1.67 1.67 1.67 ... $ Chaetodontidae : num 1 1 1 1 1 1 1 1 1 1 ... $ All.Corals : num 0 0 0 1 2 1 1 0 1 2 ... $ Treatment : chr "Caged" "Uncaged" "Caged" "Partial" ... $ Tile : chr "10" "10" "11" "11" ...

My code looks like this:

psem.model <- piecewiseSEM::psem( lm(Herbivores ~ Shark_Sum_Abundance + Mesopredators, data=sem.df), lm(Mesopredators ~ Shark_Sum_Abundance, data=sem.df), lm(Chaetodontidae ~ Shark_Sum_Abundance + Mesopredators, data=sem.df), lm(All.Corals ~ Herbivores + Chaetodontidae, data=sem.df))

The output looks like this:

summary(psem.model) |===================================================================| 100%

Structural Equation Model of psem.model

Call: Herbivores ~ Shark_Sum_Abundance + Mesopredators Mesopredators ~ Shark_Sum_Abundance Chaetodontidae ~ Shark_Sum_Abundance + Mesopredators All.Corals ~ Herbivores + Chaetodontidae

AIC 5751.800

Tests of directed separation:

No independence claims present. Tests of directed separation not possible.

--

Global goodness-of-fit:

Chi-Squared = 67.053 with P-value = 0 and on 3 degrees of freedom Fisher's C = 73.048 with P-value = 0 and on 6 degrees of freedom

Coefficients:

No independence claims present. Tests of directed separation not possible.

Signif. codes: 0 '' 0.001 '' 0.01 '' 0.05

Individual R-squared:

No independence claims present. Tests of directed separation not possible.

I'm currently using version 2.3.0 of piecewise SEM.

Thank you! Jamie

mmoroti commented 1 year ago

I have the same problem and I don't know why.

jslefche commented 1 year ago

Hmmm, I can’t seem to reproduce this. You definitely should have independence claims (3 by my count). When I make up some data and run it through, I get the following output. If you can provide a snippet of data to reproduce, I will dig in !

sem.df <- data.frame(
  Herbivores = runif(50),
  Shark_Sum_Abundance = runif(50),
  Mesopredators = runif(50),
  Chaetodontidae = runif(50),
  All.Corals = runif(50)

)

psem.model <- piecewiseSEM::psem(
  lm(Herbivores ~ Shark_Sum_Abundance + Mesopredators, data=sem.df),
  lm(Mesopredators ~ Shark_Sum_Abundance, data=sem.df),
  lm(Chaetodontidae ~ Shark_Sum_Abundance + Mesopredators, data=sem.df),
  lm(All.Corals ~ Herbivores + Chaetodontidae, data=sem.df))

piecewiseSEM::summary(psem.model)

# Structural Equation Model of psem.model
#
# Call:
#   Herbivores ~ Shark_Sum_Abundance + Mesopredators
# Mesopredators ~ Shark_Sum_Abundance
# Chaetodontidae ~ Shark_Sum_Abundance + Mesopredators
# All.Corals ~ Herbivores + Chaetodontidae
#
# AIC
# 89.273
#
# ---
#   Tests of directed separation:
#
#   Independ.Claim Test.Type DF Crit.Value P.Value
# All.Corals ~ Shark_Sum_Abundance + ...      coef 46     2.2611  0.0285 *
#   Chaetodontidae ~ Herbivores + ...      coef 46     0.6142  0.5421
# All.Corals ~ Mesopredators + ...      coef 45    -1.2862  0.2049
#
# --
#   Global goodness-of-fit:
#
#   Chi-Squared = 7.483 with P-value = 0.058 and on 3 degrees of freedom
# Fisher's C = 11.508 with P-value = 0.074 and on 6 degrees of freedom
#
# ---
# Coefficients:
#
#         Response           Predictor Estimate Std.Error DF Crit.Value P.Value Std.Estimate
#       Herbivores Shark_Sum_Abundance  -0.0805    0.1520 47    -0.5298  0.5987      -0.0770
#       Herbivores       Mesopredators   0.0388    0.1305 47     0.2971  0.7677       0.0432
#    Mesopredators Shark_Sum_Abundance  -0.0324    0.1681 48    -0.1929  0.8479      -0.0278
#   Chaetodontidae Shark_Sum_Abundance  -0.0332    0.1990 47    -0.1667  0.8683      -0.0243
#   Chaetodontidae       Mesopredators  -0.0070    0.1708 47    -0.0410  0.9674      -0.0060
#       All.Corals          Herbivores  -0.0879    0.1720 47    -0.5109  0.6118      -0.0746
#       All.Corals      Chaetodontidae   0.0205    0.1318 47     0.1554  0.8772       0.0227
#
#   Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05
#
# ---
# Individual R-squared:
#
#         Response method R.squared
#       Herbivores   none      0.01
#    Mesopredators   none      0.00
#   Chaetodontidae   none      0.00
#       All.Corals   none      0.01

Jonathan S. Lefcheck, Ph.D. Tennenbaum Coordinating Scientist MarineGEO: https://marinegeo.si.edu/ Smithsonian Institution Phone: +1 (443) 482-2443 www.jonlefcheck.nethttp://www.jonlefcheck.net

From: Matheus @.> Sent: Wednesday, March 1, 2023 5:35 PM To: @.> Cc: @.***> Subject: Re: [jslefche/piecewiseSEM] No independence claims present. Tests of directed separation not possible. (Issue #271)

External Email - Exercise Caution

I have the same problem and I don't know why.

— Reply to this email directly, view it on GitHubhttps://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fjslefche%2FpiecewiseSEM%2Fissues%2F271%23issuecomment-1450949917&data=05%7C01%7Clefcheckj%40si.edu%7Ce40d91730d79418c72e008db1aa55121%7C989b5e2a14e44efe93b78cdd5fc5d11c%7C0%7C0%7C638133069552487643%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Jeq26VMevDkeQQEvvKcPpCT3FocOBGYfcmoJ%2FpZK6Hw%3D&reserved=0, or unsubscribehttps://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAAR4AVZI3IS54HGYFRXNKELWZ7FMTANCNFSM6AAAAAATYSK2O4&data=05%7C01%7Clefcheckj%40si.edu%7Ce40d91730d79418c72e008db1aa55121%7C989b5e2a14e44efe93b78cdd5fc5d11c%7C0%7C0%7C638133069552487643%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=bZQ%2BC%2FKoKnq7P%2FZZkCkrO4zK92gBoSeD38AN5fqHvn0%3D&reserved=0. You are receiving this because you are subscribed to this thread.Message ID: @.***>

jamiekerlin commented 1 year ago

I'm having the same issue with my data. If I check everything separately, I get values (dSep, fisherC, rsquared, coefs, etc), but when I run summary() I am getting the same output. The same issue is happening even if I try running the example code on the Keeley data.

jslefche commented 1 year ago

Jamie, have you updated to the newest version of the package? Which version of R are you running? And what OS?

Jonathan S. Lefcheck, Ph.D. Tennenbaum Coordinating Scientist MarineGEO: https://marinegeo.si.edu/ Smithsonian Institution Phone: +1 (443) 482-2443 www.jonlefcheck.nethttp://www.jonlefcheck.net

From: Jamie @.> Sent: Sunday, April 2, 2023 1:57 AM To: @.> Cc: Lefcheck, @.>; @.> Subject: Re: [jslefche/piecewiseSEM] No independence claims present. Tests of directed separation not possible. (Issue #271)

External Email - Exercise Caution

I'm having the same issue with my data. If I check everything separately, I get values (dSep, fisherC, rsquared, coefs, etc), but when I run summary() I am getting the same output.

— Reply to this email directly, view it on GitHubhttps://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fjslefche%2FpiecewiseSEM%2Fissues%2F271%23issuecomment-1493232103&data=05%7C01%7Clefcheckj%40si.edu%7C11c1fca1704c44409e4408db333f2627%7C989b5e2a14e44efe93b78cdd5fc5d11c%7C0%7C0%7C638160118560780388%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=V1mriN2ptBKdybXoIzYD%2FY%2B9vncMsYT7apsnU1mZMmM%3D&reserved=0, or unsubscribehttps://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAAR4AVYC6Y2QHLFMXNTCKOTW7EIMXANCNFSM6AAAAAATYSK2O4&data=05%7C01%7Clefcheckj%40si.edu%7C11c1fca1704c44409e4408db333f2627%7C989b5e2a14e44efe93b78cdd5fc5d11c%7C0%7C0%7C638160118560780388%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=u5FAVaauyr8ocXsSybOrDw4lllMzypms%2FG1h62zn7ek%3D&reserved=0. You are receiving this because you commented.Message ID: @.***>

doinamani commented 1 year ago

Hi Jon,

I'm currently running into the same problem, using piecewiseSEM version 2.3.0 in R version 4.3.1. It doesn't matter what model I try, any version I get the message that there are no independence claims. If I run this example code, I get the output listed below:

Create fake data

set.seed(1)

data <- data.frame( x = runif(100), y1 = runif(100), y2 = runif(100), y3 = runif(100))

Create SEM using psem

modelList <- psem( lm(y1 ~ x, data), glm(y2 ~ x, data), lm(y3 ~ y1 + y2, data), data)

and I get the following output:

Structural Equation Model of modelList

Call: y1 ~ x y2 ~ x y3 ~ y1 + y2

AIC

356.819

Tests of directed separation:

No independence claims present. Tests of directed separation not possible.

Global goodness-of-fit:

Chi-Squared = 3.916 with P-value = 0.141 and on 2 degrees of freedom Fisher's C = 6.691 with P-value = 0.153 and on 4 degrees of freedom

Coefficients:

No independence claims present. Tests of directed separation not possible.

Signif. codes: 0 '' 0.001 '' 0.01 '' 0.05

Individual R-squared:

No independence claims present. Tests of directed separation not possible.

jslefche commented 1 year ago

Hmm, I cannot seem to reproduce this. Running your code below, I get:


---
Tests of directed separation:

  Independ.Claim Test.Type DF Crit.Value P.Value
    y3 ~ x + ...      coef 96    -1.8430  0.0684
   y2 ~ y1 + ...      coef 97    -0.7076  0.4792

--
Global goodness-of-fit:

Chi-Squared = 3.916 with P-value = 0.271 and on 3 degrees of freedom
Fisher's C = 6.836 with P-value = 0.145 and on 4 degrees of freedom

Have you updated R, the package, and any dependencies to the latest versions??

Cheers,

Jon

Jonathan Lefcheck, Ph.D.

Research Scientist

Integration and Application Network

University of Maryland Center for Environmental Science

www.jonlefcheck.nethttp://www.jonlefcheck.net

From: doinamani @.> Sent: Thursday, November 23, 2023 10:55 AM To: jslefche/piecewiseSEM @.> Cc: Jon Lefcheck @.>; Comment @.> Subject: Re: [jslefche/piecewiseSEM] No independence claims present. Tests of directed separation not possible. (Issue #271)

Hi Jon,

I'm currently running into the same problem, using piecewiseSEM version 2.3.0 in R version 4.3.1. It doesn't matter what model I try, any version I get the message that there are no independence claims. If I run this example code, I get the output listed below:

Create fake data

set.seed(1)

data <- data.frame( x = runif(100), y1 = runif(100), y2 = rpois(100, 1), y3 = runif(100))

Create SEM using psem

modelList <- psem( lm(y1 ~ x, data), glm(y2 ~ x, "poisson", data), lm(y3 ~ y1 + y2, data), data)

Run summary, Address conflict using conserve = T

summary(modelList, conserve = T)

and I get the following output

Structural Equation Model of modelList

Call: y1 ~ x y2 ~ x y3 ~ y1 + y2

AIC

320.711

Tests of directed separation:

No independence claims present. Tests of directed separation not possible.

-- Global goodness-of-fit:

Chi-Squared = 3.916 with P-value = 0.271 and on 3 degrees of freedom Fisher's C = 6.836 with P-value = 0.145 and on 4 degrees of freedom

Coefficients:

No independence claims present. Tests of directed separation not possible.

Signif. codes: 0 '' 0.001 '' 0.01 '' 0.05

Individual R-squared:

No independence claims present. Tests of directed separation not possible.

— Reply to this email directly, view it on GitHubhttps://www.google.com/url?q=https://github.com/jslefche/piecewiseSEM/issues/271%23issuecomment-1824652912&source=gmail-imap&ust=1701359722000000&usg=AOvVaw3eJNEH7ntCF-A3HzX9luy_, or unsubscribehttps://www.google.com/url?q=https://github.com/notifications/unsubscribe-auth/AAR4AV6L6SSHSOQ6476ZDF3YF5WWTAVCNFSM6AAAAAATYSK2O6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMRUGY2TEOJRGI&source=gmail-imap&ust=1701359722000000&usg=AOvVaw08WHeOi3wmNTrp_XiQeLNk. You are receiving this because you commented.Message ID: @.***>

zpinocchio commented 10 months ago

I had the same problem.

However, I found that in a pure environment, summary function works fine with all results, such as New Session. Maybe there's name space contamination in there?

doinamani commented 9 months ago

I was initially working in an Rmarkdown file, but when I gave up on that idea and started working in a regular R script the error disappeared. So it seems using Rmarkdown creates some issues as well

marissadyck commented 3 weeks ago

I had the same issue while using version 2.3.0 and R version 4.2.3 working in Rmarkdown and when I changed to a normal script file the issue was resolved as well