search

jacob-long / jtools

Tools for summarizing/visualizing regressions and other helpful stuff
https://jtools.jacob-long.com
GNU General Public License v3.0
164 stars 22 forks source link

Johnson-Neyman technique with multilevel equations #11

Closed seonghobae closed 6 years ago

seonghobae commented 6 years ago

Hi,

I happy to meet this amazing library to run Johnson-Neyman makes easily.

However, I have a trouble to run in multilevel equations.

Here is the pseudo code:

  library('lme4')
  modJN1 <- glmer(IMP ~ ACQ + DEF + (ACQ+DEF|teamid) + RADIC + INCRE +
                    RADIC:ACQ + RADIC:DEF + INCRE:ACQ + INCRE:DEF,
                  data = rawSEM_recursive, REML = T)
  sim_slopes(modJN1, pred = RADIC, modx = ACQ, jnplot = TRUE)

As the result, I got the negative hessian error messages. If I have to make a Johnson-Neyman with Multilevel equations, What I have to do?

Best, Seongho

seonghobae commented 6 years ago

Here is a error messages

> sim_slopes(modJN1, pred = RADIC, modx = ACQ, jnplot = TRUE)
Error in jns[[1]]$plot : $ operator is invalid for atomic vectors
추가정보: 경고메시지(들): 
1: In checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv,  :
  unable to evaluate scaled gradient
2: In checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv,  :
  Model failed to converge: degenerate  Hessian with 1 negative eigenvalues
seonghobae commented 6 years ago

I tried to calibrate without the random effect, however, it fails again.

> modJN2 <- glmer(IMP ~ ACQ + DEF + RADIC + INCRE + RADIC:ACQ + RADIC:DEF + INCRE:ACQ + INCRE:DEF, data = rawSEM_recursive, REML = F)
에러: No random effects terms specified in formula
추가정보: 경고메시지(들): 
In glmer(IMP ~ ACQ + DEF + RADIC + INCRE + RADIC:ACQ + RADIC:DEF +  :
  calling glmer() with family=gaussian (identity link) as a shortcut to lmer() is deprecated; please call lmer() directly
> sim_slopes(modJN2, pred = RADIC, modx = ACQ, jnplot = TRUE)
에러: $ operator is invalid for atomic vectors
추가정보: 경고메시지(들): 
1: In pp$ptr() : restarting interrupted promise evaluation
2: In pp$ptr() : internal error -3 in R_decompress1
3: In checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv,  :
  unable to evaluate scaled gradient
4: In checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv,  :
  Model failed to converge: degenerate  Hessian with 1 negative eigenvalues
5: In checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv,  :
  unable to evaluate scaled gradient
6: In checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv,  :
  Model failed to converge: degenerate  Hessian with 1 negative eigenvalues
jacob-long commented 6 years ago

I haven't been able to replicate this particular error, but in general I had not tested johnson_neyman with mixed models. I just published an update with a few tweaks that should add compatibility, but in my testing I never generated these convergence errors so I can't be sure where they come from/if they were fixed.

To see if the latest update fixed this, install the development version from Github:

install.packages("devtools")
devtools::install_github("jacob-long/jtools")
seonghobae commented 6 years ago

Okay, cool! I've check out your work. jtools::johnson_neyman is works. That's useful to me. However, I also need to get results from 'sim__slopes'. Could you Have any ideas to get results of simple slopes in sim_slopes even that is trick?

> jtools::johnson_neyman(mod1_3_freq,  pred = RADIC, modx = DEF)
JOHNSON-NEYMAN INTERVAL

The slope of RADIC is p < .05 when DEF is INSIDE this interval:
[-4.41, -0.34]
Note: The range of observed values of DEF is [-1.81, 2.62]
> jtools::sim_slopes(mod1_3_freq,  pred = RADIC, modx = DEF)
Error: $ operator is invalid for atomic vectors

Best, Seongho

jacob-long commented 6 years ago

Hello Seongho, I believe I've fixed this bug. Please download the latest Github version and let me know if it's working for you.

seonghobae commented 6 years ago

Okay, I installed new version but I still have a trouble to run.

> modSimSlope1 <- jtools::sim_slopes(mod1_lme, pred = RADIC, modx = ACQ)
경고메시지(들): 
1: In checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv,  :
  unable to evaluate scaled gradient
2: In checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv,  :
  Model failed to converge: degenerate  Hessian with 3 negative eigenvalues
3: In checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv,  :
  unable to evaluate scaled gradient
4: In checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv,  :
  Model failed to converge: degenerate  Hessian with 1 negative eigenvalues
5: In checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv,  :
  unable to evaluate scaled gradient
6: In checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv,  :
  Model failed to converge: degenerate  Hessian with 1 negative eigenvalues
7: In checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv,  :
  unable to evaluate scaled gradient
8: In checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv,  :
  Model failed to converge: degenerate  Hessian with 2 negative eigenvalues
> modSimSlope1
JOHNSON-NEYMAN INTERVAL

The slope of RADIC is p < .05 when ACQ is INSIDE this interval:
[0.21, 2.98]
Note: The range of observed values of ACQ is [-3.19, 3.03]

SIMPLE SLOPES ANALYSIS

Slope of RADIC when ACQ = 0.95 (+ 1 SD): 
Est. S.E.    p 
0.25 0.11 0.05 

Slope of RADIC when ACQ = -0.02 (Mean): 
Est. S.E.    p 
0.15 0.09 0.13 

Slope of RADIC when ACQ = -0.99 (- 1 SD): 
Est. S.E.    p 
0.06 0.12 0.63 

> sessionInfo()
R version 3.4.2 (2017-09-28)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 17.10

Matrix products: default
BLAS: /opt/microsoft/ropen/3.4.2/lib64/R/lib/libRblas.so
LAPACK: /opt/microsoft/ropen/3.4.2/lib64/R/lib/libRlapack.so

locale:
 [1] LC_CTYPE=ko_KR.UTF-8       LC_NUMERIC=C               LC_TIME=ko_KR.UTF-8        LC_COLLATE=ko_KR.UTF-8    
 [5] LC_MONETARY=ko_KR.UTF-8    LC_MESSAGES=ko_KR.UTF-8    LC_PAPER=ko_KR.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C             LC_MEASUREMENT=ko_KR.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] jtools_0.9.2         RevoUtils_10.0.6     RevoUtilsMath_10.0.1

loaded via a namespace (and not attached):
  [1] uuid_0.1-2            blme_1.0-4            plyr_1.8.4            igraph_1.1.2          GPArotation_2014.11-1
  [6] lazyeval_0.2.1        TMB_1.7.12            splines_3.4.2         crosstalk_1.0.0       listenv_0.7.0        
 [11] ggplot2_2.2.1         TH.data_1.0-8         rstantools_1.4.0      inline_0.3.14         digest_0.6.14        
 [16] htmltools_0.3.6       rsconnect_0.8.5       memoise_1.1.0         magrittr_1.5          cluster_2.0.6        
 [21] NCmisc_1.1.5          globals_0.11.0-9000   brms_2.1.0            modelr_0.1.1          matrixStats_0.53.0   
 [26] R.utils_2.6.0         officer_0.2.1         xts_0.10-1            sandwich_2.4-0        prettyunits_1.0.2    
 [31] colorspace_1.3-2      haven_1.1.1           dplyr_0.7.4           crayon_1.3.4          lme4_1.1-15          
 [36] bindr_0.1             survival_2.41-3       zoo_1.8-1             glue_1.2.0            gtable_0.2.0         
 [41] emmeans_1.1           sjstats_0.14.0        sjmisc_2.6.3          mirt_1.26.7           rstan_2.17.3         
 [46] abind_1.4-5           scales_0.5.0          mvtnorm_1.0-6         ggeffects_0.3.1       reghelper_0.3.3      
 [51] miniUI_0.1.1          Rcpp_0.12.15          merTools_0.4.1        xtable_1.8-2          progress_1.1.2       
 [56] kaefa_0.1.155         foreign_0.8-69        stats4_3.4.2          prediction_0.2.0      StanHeaders_2.17.2   
 [61] survey_3.33           DT_0.3                httr_1.3.1            htmlwidgets_1.0       threejs_0.3.1        
 [66] modeltools_0.2-21     pkgconfig_2.0.1       loo_1.1.0             R.methodsS3_1.7.1     nnet_7.3-12          
 [71] labeling_0.3          tidyselect_0.2.3      rlang_0.1.6           reshape2_1.4.3        munsell_0.4.3        
 [76] tools_3.4.2           cli_1.0.0             sjlabelled_1.0.6      devtools_1.13.4       broom_0.4.3          
 [81] stringr_1.2.0         yaml_2.1.16           arm_1.9-3             knitr_1.18            zip_1.0.0            
 [86] purrr_0.2.4           bindrcpp_0.2          coin_1.2-2            future_1.6.2-9000     nlme_3.1-131         
 [91] mime_0.5              proftools_0.99-2      R.oo_1.21.0           xml2_1.2.0            pbkrtest_0.4-7       
 [96] compiler_3.4.2        bayesplot_1.4.0       shinythemes_1.1.1     rstudioapi_0.7        curl_3.1             
[101] tibble_1.4.2          stringi_1.1.6         Brobdingnag_1.2-4     forcats_0.2.0         lattice_0.20-35      
[106] Matrix_1.2-12         psych_1.7.8           nloptr_1.0.4          markdown_0.8          shinyjs_1.0          
[111] vegan_2.4-6           permute_0.9-4         effects_4.0-0         stringdist_0.9.4.6    pillar_1.1.0         
[116] pwr_1.2-1             lmtest_0.9-35         bridgesampling_0.4-0  estimability_1.2      httpuv_1.3.5         
[121] R6_2.2.2              gridExtra_2.3         codetools_0.2-15      colourpicker_1.0      MASS_7.3-48          
[126] gtools_3.5.0          assertthat_0.2.0      withr_2.1.1           shinystan_2.4.0       mnormt_1.5-5         
[131] Deriv_3.8.3           multcomp_1.4-8        mgcv_1.8-23           parallel_3.4.2        grid_3.4.2           
[136] sjPlot_2.4.0          tidyr_0.7.2           coda_0.19-1           glmmTMB_0.2.0         minqa_1.2.4          
[141] snakecase_0.8.1       carData_3.0-0         git2r_0.21.0          shiny_1.0.5           base64enc_0.1-3      
[146] dygraphs_1.1.1.4
seonghobae commented 6 years ago

Finally, I changed optimiser settings. It works! Thanks, Jacob!

Best, Seongho

rnorouzian commented 3 years ago

It would be great if the package could possibly accept lme() models from the library(nlme) as well, any plans to include that?