konfound function keeps running, not returning results

[Ritazheng2]

I was trying to run the konfound command for the fixed effects model, but it kept running not returning any results. It only shows this:

Registered S3 method overwritten by 'broom.mixed': method from tidy.gamlss broom

Any recommendations? Thanks!

[jrosen48]

Hi, can you share the code that you are running? Also, can you run sessionInfo() and share that, too? Thank you!

[Ritazheng2]

Hi, can you share the code that you are running? Also, can you run sessionInfo() and share that, too? Thank you!

Thanks for the reply. Here's the code I am running: library(pbkrtest) if (requireNamespace("lme4")) { library(lme4) mle<-lmer(position_up ~ relevel(mob, ref=3)+ position_level+inter_circle_mob +intra_mob+ previous_years_with_coach +dest_previous5yearstrengthrankin +previous5yearsteamstrengthrankin+relativescoringoffense+ relativescoringdefense+ pointsperpossessionoffense +pointsperpossessiondefense+ total_schools_coached+ total_coaches_coached_with+ dest_relativescoringoffense +dest_relativescoringdefense+ dest_pointsperpossessionoffense +dest_pointsperpossessiondefense+year+team_id +(1|coach_id), data = data) konfound_output<-konfound(mle, mob, to_return = c("raw_output", "thresh_plot", "corr_plot")) }

Here's the result returned by the sessionInfo():

sessionInfo() R version 4.0.3 (2020-10-10) Platform: x86_64-w64-mingw32/x64 (64-bit) Running under: Windows 10 x64 (build 18362)

Matrix products: default

locale: [1] LC_COLLATE=English_United States.1252 [2] LC_CTYPE=English_United States.1252
[3] LC_MONETARY=English_United States.1252 [4] LC_NUMERIC=C
[5] LC_TIME=English_United States.1252

attached base packages: [1] stats graphics grDevices utils datasets methods base

other attached packages: [1] pbkrtest_0.4-8.6 konfound_0.2.2 lme4_1.1-26 Matrix_1.2-18

loaded via a namespace (and not attached): [1] Rcpp_1.0.5 plyr_1.8.6 pillar_1.4.7
[4] compiler_4.0.3 nloptr_1.2.2.2 TMB_1.7.18
[7] tools_4.0.3 boot_1.3-25 statmod_1.4.35
[10] lifecycle_0.2.0 tibble_3.0.4 nlme_3.1-149
[13] lattice_0.20-41 pkgconfig_2.0.3 rlang_0.4.9
[16] rstudioapi_0.13 parallel_4.0.3 coda_0.19-4
[19] stringr_1.4.0 dplyr_1.0.2 generics_0.1.0
[22] vctrs_0.3.5 grid_4.0.3 tidyselect_1.1.0 [25] glue_1.4.2 R6_2.5.0 minqa_1.2.4
[28] reshape2_1.4.4 purrr_0.3.4 tidyr_1.1.2
[31] magrittr_2.0.1 backports_1.2.0 ellipsis_0.3.1
[34] MASS_7.3-53 splines_4.0.3 broom.mixed_0.2.6 [37] stringi_1.5.3 broom_0.7.2 crayon_1.3.4

Thank you!

[jrosen48]

Will take a peak tomorrow

Get Outlook for Androidhttps://aka.ms/ghei36

---

From: Ritazheng2 notifications@github.com Sent: Wednesday, December 9, 2020 8:51:23 PM To: jrosen48/konfound konfound@noreply.github.com Cc: Rosenberg, Joshua jmrosenberg@utk.edu; Comment comment@noreply.github.com Subject: Re: [jrosen48/konfound] konfound function keeps running, not returning results (#37)

[External Email]

Hi, can you share the code that you are running? Also, can you run sessionInfo() and share that, too? Thank you!

Thanks for the reply. Here's the code I am running: library(pbkrtest) if (requireNamespace("lme4")) { library(lme4) mle<-lmer(position_up ~ relevel(mob, ref=3)+ position_level+inter_circle_mob +intra_mob+ previous_years_with_coach +dest_previous5yearstrengthrankin +previous5yearsteamstrengthrankin+relativescoringoffense+ relativescoringdefense+ pointsperpossessionoffense +pointsperpossessiondefense+ total_schools_coached+ total_coaches_coached_with+ dest_relativescoringoffense +dest_relativescoringdefense+ dest_pointsperpossessionoffense +dest_pointsperpossessiondefense+year+team_id +(1|coach_id), data = data) konfound_output<-konfound(mle, mob, to_return = c("raw_output", "thresh_plot", "corr_plot")) }

Here's the result returned by the sessionInfo():

sessionInfo() R version 4.0.3 (2020-10-10) Platform: x86_64-w64-mingw32/x64 (64-bit) Running under: Windows 10 x64 (build 18362)

Matrix products: default

locale: [1] LC_COLLATE=English_United States.1252 [2] LC_CTYPE=English_United States.1252 [3] LC_MONETARY=English_United States.1252 [4] LC_NUMERIC=C [5] LC_TIME=English_United States.1252

attached base packages: [1] stats graphics grDevices utils datasets methods base

other attached packages: [1] pbkrtest_0.4-8.6 konfound_0.2.2 lme4_1.1-26 Matrix_1.2-18

loaded via a namespace (and not attached): [1] Rcpp_1.0.5 plyr_1.8.6 pillar_1.4.7 [4] compiler_4.0.3 nloptr_1.2.2.2 TMB_1.7.18 [7] tools_4.0.3 boot_1.3-25 statmod_1.4.35 [10] lifecycle_0.2.0 tibble_3.0.4 nlme_3.1-149 [13] lattice_0.20-41 pkgconfig_2.0.3 rlang_0.4.9 [16] rstudioapi_0.13 parallel_4.0.3 coda_0.19-4 [19] stringr_1.4.0 dplyr_1.0.2 generics_0.1.0 [22] vctrs_0.3.5 grid_4.0.3 tidyselect_1.1.0 [25] glue_1.4.2 R6_2.5.0 minqa_1.2.4 [28] reshape2_1.4.4 purrr_0.3.4 tidyr_1.1.2 [31] magrittr_2.0.1 backports_1.2.0 ellipsis_0.3.1 [34] MASS_7.3-53 splines_4.0.3 broom.mixed_0.2.6 [37] stringi_1.5.3 broom_0.7.2 crayon_1.3.4

Thank you!

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/jrosen48/konfound/issues/37#issuecomment-742180723, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABDCD5QMLLFKMCLNQ2JGHHDSUASRXANCNFSM4UUKGH4A.

[jrosen48]

I checked and think this is working in the latest version. Can you please run:

install.packages("devtools")
devtools::install_github("jrosen48/konfound")

And then re-run.

[Ritazheng2]

I checked and think this is working in the latest version. Can you please run:

install.packages("devtools")
devtools::install_github("jrosen48/konfound")

And then re-run.

Thanks for your help, Joshua. I tried the code and reran the test. It's still running, without returning the results. I also reinstalled the konfound package. I am using the same code as before and here's the result returned from sessionInfo(): R version 4.0.3 (2020-10-10) Platform: x86_64-w64-mingw32/x64 (64-bit) Running under: Windows 10 x64 (build 18362)

Matrix products: default

locale: [1] LC_COLLATE=English_United States.1252 [2] LC_CTYPE=English_United States.1252
[3] LC_MONETARY=English_United States.1252 [4] LC_NUMERIC=C
[5] LC_TIME=English_United States.1252

attached base packages: [1] stats graphics grDevices utils datasets methods base

other attached packages: [1] konfound_0.2.2 lme4_1.1-26 Matrix_1.2-18

loaded via a namespace (and not attached): [1] Rcpp_1.0.5 pillar_1.4.7 nloptr_1.2.2.2
[4] compiler_4.0.3 prettyunits_1.1.1 remotes_2.2.0
[7] tools_4.0.3 boot_1.3-25 testthat_3.0.0
[10] digest_0.6.27 pkgbuild_1.1.0 pkgload_1.1.0
[13] statmod_1.4.35 tibble_3.0.4 memoise_1.1.0
[16] lifecycle_0.2.0 nlme_3.1-149 lattice_0.20-41
[19] pkgconfig_2.0.3 rlang_0.4.9 cli_2.2.0
[22] rstudioapi_0.13 dplyr_1.0.2 withr_2.3.0
[25] generics_0.1.0 vctrs_0.3.5 desc_1.2.0
[28] fs_1.5.0 devtools_2.3.2 tidyselect_1.1.0 [31] rprojroot_2.0.2 grid_4.0.3 glue_1.4.2
[34] R6_2.5.0 processx_3.4.5 fansi_0.4.1
[37] sessioninfo_1.1.1 minqa_1.2.4 callr_3.5.1
[40] purrr_0.3.4 magrittr_2.0.1 ps_1.5.0
[43] ellipsis_0.3.1 usethis_2.0.0 splines_4.0.3
[46] MASS_7.3-53 assertthat_0.2.1 crayon_1.3.4

[jrosen48]

Sure thing. For how long was it running? This may be an issue with how konfound works for multi-level models. It calculates the Kenward-Rogers degrees of freedom and can take awhile for some models (those with a large dataset or many parameters). Can you run the example with an lme4 model (?konfound) to confirm that works? That should run in 1s or so.

From: Ritazheng2 notifications@github.com Date: Friday, December 11, 2020 at 12:09 PM To: jrosen48/konfound konfound@noreply.github.com Cc: Rosenberg, Joshua jmrosenberg@utk.edu, Comment comment@noreply.github.com Subject: Re: [jrosen48/konfound] konfound function keeps running, not returning results (#37)

[External Email]

I checked and think this is working in the latest version. Can you please run:

install.packages("devtools")

devtools::install_github("jrosen48/konfound")

And then re-run.

Thanks for your help, Joshua. I tried the code and reran the test. It's still running, without returning the results. I also reinstalled the konfound package. I am using the same code as before and here's the result returned from sessionInfo(): R version 4.0.3 (2020-10-10) Platform: x86_64-w64-mingw32/x64 (64-bit) Running under: Windows 10 x64 (build 18362)

Matrix products: default

locale: [1] LC_COLLATE=English_United States.1252 [2] LC_CTYPE=English_United States.1252 [3] LC_MONETARY=English_United States.1252 [4] LC_NUMERIC=C [5] LC_TIME=English_United States.1252

attached base packages: [1] stats graphics grDevices utils datasets methods base

other attached packages: [1] konfound_0.2.2 lme4_1.1-26 Matrix_1.2-18

loaded via a namespace (and not attached): [1] Rcpp_1.0.5 pillar_1.4.7 nloptr_1.2.2.2 [4] compiler_4.0.3 prettyunits_1.1.1 remotes_2.2.0 [7] tools_4.0.3 boot_1.3-25 testthat_3.0.0 [10] digest_0.6.27 pkgbuild_1.1.0 pkgload_1.1.0 [13] statmod_1.4.35 tibble_3.0.4 memoise_1.1.0 [16] lifecycle_0.2.0 nlme_3.1-149 lattice_0.20-41 [19] pkgconfig_2.0.3 rlang_0.4.9 cli_2.2.0 [22] rstudioapi_0.13 dplyr_1.0.2 withr_2.3.0 [25] generics_0.1.0 vctrs_0.3.5 desc_1.2.0 [28] fs_1.5.0 devtools_2.3.2 tidyselect_1.1.0 [31] rprojroot_2.0.2 grid_4.0.3 glue_1.4.2 [34] R6_2.5.0 processx_3.4.5 fansi_0.4.1 [37] sessioninfo_1.1.1 minqa_1.2.4 callr_3.5.1 [40] purrr_0.3.4 magrittr_2.0.1 ps_1.5.0 [43] ellipsis_0.3.1 usethis_2.0.0 splines_4.0.3 [46] MASS_7.3-53 assertthat_0.2.1 crayon_1.3.4

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/jrosen48/konfound/issues/37#issuecomment-743315309, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABDCD5TWEFDQDG4T2S5RIL3SUJG4NANCNFSM4UUKGH4A.

[Ritazheng2]

Sure thing. For how long was it running? This may be an issue with how konfound works for multi-level models. It calculates the Kenward-Rogers degrees of freedom and can take awhile for some models (those with a large dataset or many parameters). Can you run the example with an lme4 model (?konfound) to confirm that works? That should run in 1s or so. From: Ritazheng2 notifications@github.com Date: Friday, December 11, 2020 at 12:09 PM To: jrosen48/konfound konfound@noreply.github.com Cc: Rosenberg, Joshua jmrosenberg@utk.edu, Comment comment@noreply.github.com Subject: Re: [jrosen48/konfound] konfound function keeps running, not returning results (#37) [External Email] I checked and think this is working in the latest version. Can you please run: install.packages("devtools") devtools::install_github("jrosen48/konfound") And then re-run. Thanks for your help, Joshua. I tried the code and reran the test. It's still running, without returning the results. I also reinstalled the konfound package. I am using the same code as before and here's the result returned from sessionInfo(): R version 4.0.3 (2020-10-10) Platform: x86_64-w64-mingw32/x64 (64-bit) Running under: Windows 10 x64 (build 18362) Matrix products: default locale: [1] LC_COLLATE=English_United States.1252 [2] LC_CTYPE=English_United States.1252 [3] LC_MONETARY=English_United States.1252 [4] LC_NUMERIC=C [5] LC_TIME=English_United States.1252 attached base packages: [1] stats graphics grDevices utils datasets methods base other attached packages: [1] konfound_0.2.2 lme4_1.1-26 Matrix_1.2-18 loaded via a namespace (and not attached): [1] Rcpp_1.0.5 pillar_1.4.7 nloptr_1.2.2.2 [4] compiler_4.0.3 prettyunits_1.1.1 remotes_2.2.0 [7] tools_4.0.3 boot_1.3-25 testthat_3.0.0 [10] digest_0.6.27 pkgbuild_1.1.0 pkgload_1.1.0 [13] statmod_1.4.35 tibble_3.0.4 memoise_1.1.0 [16] lifecycle_0.2.0 nlme_3.1-149 lattice_0.20-41 [19] pkgconfig_2.0.3 rlang_0.4.9 cli_2.2.0 [22] rstudioapi_0.13 dplyr_1.0.2 withr_2.3.0 [25] generics_0.1.0 vctrs_0.3.5 desc_1.2.0 [28] fs_1.5.0 devtools_2.3.2 tidyselect_1.1.0 [31] rprojroot_2.0.2 grid_4.0.3 glue_1.4.2 [34] R6_2.5.0 processx_3.4.5 fansi_0.4.1 [37] sessioninfo_1.1.1 minqa_1.2.4 callr_3.5.1 [40] purrr_0.3.4 magrittr_2.0.1 ps_1.5.0 [43] ellipsis_0.3.1 usethis_2.0.0 splines_4.0.3 [46] MASS_7.3-53 assertthat_0.2.1 crayon_1.3.4 — You are receiving this because you commented. Reply to this email directly, view it on GitHub<#37 (comment)>, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABDCD5TWEFDQDG4T2S5RIL3SUJG4NANCNFSM4UUKGH4A.

Joshua, thanks. I had waited for around 20 minutes and then I stopped it. Here's the result from the example in the package. The example works. I might need to wait longer and see how it goes.

if (requireNamespace("lme4")) {

• library(lme4)
• m3 <- fm1 <- lme4::lmer(Reaction ~ Days + (1 | Subject), sleepstudy)
• konfound(m3, Days)
• } Percent Bias Necessary to Invalidate the Inference: To invalidate an inference, 84.83% of the estimate would have to be due to bias. This is based on a threshold of 1.588 for statistical significance (alpha = 0.05). To invalidate an inference, 137 observations would have to be replaced with cases for which the effect is 0. See Frank et al. (2013) for a description of the method Citation: Frank, K.A., Maroulis, S., Duong, M., and Kelcey, B. 2013. What would it take to change an inference? Using Rubin's causal model to interpret the robustness of causal inferences. Education, Evaluation and Policy Analysis, 35 437-460. Impact Threshold for a Confounding Variable: The minimum impact to invalidate an inference for a null hypothesis of 0 effect is based on a correlation of 0.817 with the outcome and at 0.817 with the predictor of interest (conditioning on observed covariates) based on a threshold of 0.155 for statistical significance (alpha = 0.05). Correspondingly the impact of an omitted variable (as defined in Frank 2000) must be 0.817 X 0.817 = 0.667 to invalidate an inference for a null hypothesis of 0 effect. See Frank (2000) for a description of the method Citation: Frank, K. 2000. Impact of a confounding variable on the inference of a regression coefficient. Sociological Methods and Research, 29 (2), 147-194 Note that the Kenward-Roger approximation is used to estimate degrees of freedom for the predictor(s) of interest. We are presently working to add other methods for calculating the degrees of freedom for the predictor(s) of interest. If you wish to use other methods now, consider those detailed here: https://bbolker.github.io/mixedmodels-misc/glmmFAQ.html#why-doesnt-lme4-display-denominator-degrees-of-freedomp-values-what-other-options-do-i-have. You can then enter degrees of freedom obtained from another method along with the coefficient, number of observations, and number of covariates to the pkonfound() function to quantify the robustness of the inference. NULL

[jrosen48]

Sounds good. We may want to build in the option for other df calculations. I will add an issue for that. It may not be too hard to do. The Satterthwaite df should be much quicker to calculate. From: Ritazheng2 notifications@github.com Date: Friday, December 11, 2020 at 12:41 PM To: jrosen48/konfound konfound@noreply.github.com Cc: Rosenberg, Joshua jmrosenberg@utk.edu, Comment comment@noreply.github.com Subject: Re: [jrosen48/konfound] konfound function keeps running, not returning results (#37)

Sure thing. For how long was it running? This may be an issue with how konfound works for multi-level models. It calculates the Kenward-Rogers degrees of freedom and can take awhile for some models (those with a large dataset or many parameters). Can you run the example with an lme4 model (?konfound) to confirm that works? That should run in 1s or so. From: Ritazheng2 notifications@github.commailto:notifications@github.com Date: Friday, December 11, 2020 at 12:09 PM To: jrosen48/konfound konfound@noreply.github.commailto:konfound@noreply.github.com Cc: Rosenberg, Joshua jmrosenberg@utk.edumailto:jmrosenberg@utk.edu, Comment comment@noreply.github.commailto:comment@noreply.github.com Subject: Re: [jrosen48/konfound] konfound function keeps running, not returning results (#37https://github.com/jrosen48/konfound/issues/37) [External Email] I checked and think this is working in the latest version. Can you please run: install.packages("devtools") devtools::install_github("jrosen48/konfound") And then re-run. Thanks for your help, Joshua. I tried the code and reran the test. It's still running, without returning the results. I also reinstalled the konfound package. I am using the same code as before and here's the result returned from sessionInfo(): R version 4.0.3 (2020-10-10) Platform: x86_64-w64-mingw32/x64 (64-bit) Running under: Windows 10 x64 (build 18362) Matrix products: default locale: [1] LC_COLLATE=English_United States.1252 [2] LC_CTYPE=English_United States.1252 [3] LC_MONETARY=English_United States.1252 [4] LC_NUMERIC=C [5] LC_TIME=English_United States.1252 attached base packages: [1] stats graphics grDevices utils datasets methods base other attached packages: [1] konfound_0.2.2 lme4_1.1-26 Matrix_1.2-18 loaded via a namespace (and not attached): [1] Rcpp_1.0.5 pillar_1.4.7 nloptr_1.2.2.2 [4] compiler_4.0.3 prettyunits_1.1.1 remotes_2.2.0 [7] tools_4.0.3 boot_1.3-25 testthat_3.0.0 [10] digest_0.6.27 pkgbuild_1.1.0 pkgload_1.1.0 [13] statmod_1.4.35 tibble_3.0.4 memoise_1.1.0 [16] lifecycle_0.2.0 nlme_3.1-149 lattice_0.20-41 [19] pkgconfig_2.0.3 rlang_0.4.9 cli_2.2.0 [22] rstudioapi_0.13 dplyr_1.0.2 withr_2.3.0 [25] generics_0.1.0 vctrs_0.3.5 desc_1.2.0 [28] fs_1.5.0 devtools_2.3.2 tidyselect_1.1.0 [31] rprojroot_2.0.2 grid_4.0.3 glue_1.4.2 [34] R6_2.5.0 processx_3.4.5 fansi_0.4.1 [37] sessioninfo_1.1.1 minqa_1.2.4 callr_3.5.1 [40] purrr_0.3.4 magrittr_2.0.1 ps_1.5.0 [43] ellipsis_0.3.1 usethis_2.0.0 splines_4.0.3 [46] MASS_7.3-53 assertthat_0.2.1 crayon_1.3.4 — You are receiving this because you commented. Reply to this email directly, view it on GitHub<#37 (comment)https://github.com/jrosen48/konfound/issues/37#issuecomment-743315309>, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABDCD5TWEFDQDG4T2S5RIL3SUJG4NANCNFSM4UUKGH4A.

Joshua, thanks. I had waited for around 20 minutes and then I stopped it. Here's the result from the example in the package. The example works. I might need to wait longer and see how it goes.

if (requireNamespace("lme4")) {

· library(lme4)

·

*

· m3 <- fm1 <- lme4::lmer(Reaction ~ Days + (1 | Subject), sleepstudy)

·

*

· konfound(m3, Days)

·

*

• } Percent Bias Necessary to Invalidate the Inference: To invalidate an inference, 84.83% of the estimate would have to be due to bias. This is based on a threshold of 1.588 for statistical significance (alpha = 0.05). To invalidate an inference, 137 observations would have to be replaced with cases for which the effect is 0. See Frank et al. (2013) for a description of the method Citation: Frank, K.A., Maroulis, S., Duong, M., and Kelcey, B. 2013. What would it take to change an inference? Using Rubin's causal model to interpret the robustness of causal inferences. Education, Evaluation and Policy Analysis, 35 437-460. Impact Threshold for a Confounding Variable: The minimum impact to invalidate an inference for a null hypothesis of 0 effect is based on a correlation of 0.817 with the outcome and at 0.817 with the predictor of interest (conditioning on observed covariates) based on a threshold of 0.155 for statistical significance (alpha = 0.05). Correspondingly the impact of an omitted variable (as defined in Frank 2000) must be 0.817 X 0.817 = 0.667 to invalidate an inference for a null hypothesis of 0 effect. See Frank (2000) for a description of the method Citation: Frank, K. 2000. Impact of a confounding variable on the inference of a regression coefficient. Sociological Methods and Research, 29 (2), 147-194 Note that the Kenward-Roger approximation is used to estimate degrees of freedom for the predictor(s) of interest. We are presently working to add other methods for calculating the degrees of freedom for the predictor(s) of interest. If you wish to use other methods now, consider those detailed here: https://bbolker.github.io/mixedmodels-misc/glmmFAQ.html#why-doesnt-lme4-display-denominator-degrees-of-freedomp-values-what-other-options-do-i-have. You can then enter degrees of freedom obtained from another method along with the coefficient, number of observations, and number of covariates to the pkonfound() function to quantify the robustness of the inference. NULL

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/jrosen48/konfound/issues/37#issuecomment-743331133, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABDCD5QEQ5G7DVDF2M5SWB3SUJKUFANCNFSM4UUKGH4A.