All models failed to converge! with vignette data

[xec-cm]

Since a few days ago, code that runs corncob that used to work with the same data now returns an error. I think it could be some modification on some corncob dependency.

I have reproduced the error with an example of the corncob vignette:

library(corncob)
data(soil_phylum_small)

soil <- soil_phylum_small

set.seed(1)
da_analysis <- differentialTest(
  formula = ~ DayAmdmt,
  phi.formula = ~ DayAmdmt,
  formula_null = ~ 1,
  phi.formula_null = ~ DayAmdmt,
  test = "Wald",
  boot = FALSE,
  data = soil,
  fdr_cutoff = 0.05
)
#> Error in differentialTest(formula = ~DayAmdmt, phi.formula = ~DayAmdmt, : All models failed to converge! 
#> 
#>            If you are seeing this, it is likely that your model is overspecified. This occurs when your sample size is not large enough to estimate all the parameters of your model. This is most commonly due to categorical variables that include many categories. 
#> 
#>            Alternatively, double-check your values for the arguments `link`, `phi.link`, and `method` to makes sure that they follow the specified options. 
#> 
#>            To confirm you have fixed the issue, try running a model for a single taxon with bbdml.

devtools::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#>  setting  value
#>  version  R version 4.1.2 (2021-11-01)
#>  os       macOS Big Sur 10.16
#>  system   x86_64, darwin17.0
#>  ui       X11
#>  language (EN)
#>  collate  es_ES.UTF-8
#>  ctype    es_ES.UTF-8
#>  tz       Europe/Madrid
#>  date     2022-08-31
#>  pandoc   2.18 @ /Applications/RStudio.app/Contents/MacOS/quarto/bin/tools/ (via rmarkdown)
#> 
#> ─ Packages ───────────────────────────────────────────────────────────────────
#>  package            * version      date (UTC) lib source
#>  ade4                 1.7-19       2022-04-19 [1] CRAN (R 4.1.2)
#>  ape                  5.6-2        2022-03-02 [1] CRAN (R 4.1.2)
#>  assertthat           0.2.1        2019-03-21 [1] CRAN (R 4.1.0)
#>  Biobase              2.54.0       2021-10-26 [1] Bioconductor
#>  BiocGenerics         0.40.0       2021-10-26 [1] Bioconductor
#>  biomformat           1.22.0       2021-10-26 [1] Bioconductor
#>  Biostrings           2.62.0       2021-10-26 [1] Bioconductor
#>  bitops               1.0-7        2021-04-24 [1] CRAN (R 4.1.0)
#>  cachem               1.0.6        2021-08-19 [1] CRAN (R 4.1.0)
#>  callr                3.7.2        2022-08-22 [1] CRAN (R 4.1.2)
#>  cli                  3.3.0        2022-04-25 [1] CRAN (R 4.1.2)
#>  cluster              2.1.4        2022-08-22 [1] CRAN (R 4.1.2)
#>  codetools            0.2-18       2020-11-04 [1] CRAN (R 4.1.2)
#>  colorspace           2.0-3        2022-02-21 [1] CRAN (R 4.1.2)
#>  corncob            * 0.2.0        2021-03-11 [1] CRAN (R 4.1.0)
#>  crayon               1.5.1        2022-03-26 [1] CRAN (R 4.1.2)
#>  data.table           1.14.2       2021-09-27 [1] CRAN (R 4.1.0)
#>  DBI                  1.1.3        2022-06-18 [1] CRAN (R 4.1.2)
#>  detectseparation     0.3          2022-08-26 [1] CRAN (R 4.1.2)
#>  devtools             2.4.4        2022-07-20 [1] CRAN (R 4.1.2)
#>  digest               0.6.29       2021-12-01 [1] CRAN (R 4.1.0)
#>  dplyr                1.0.9        2022-04-28 [1] CRAN (R 4.1.2)
#>  ellipsis             0.3.2        2021-04-29 [1] CRAN (R 4.1.0)
#>  evaluate             0.16         2022-08-09 [1] CRAN (R 4.1.2)
#>  fansi                1.0.3        2022-03-24 [1] CRAN (R 4.1.2)
#>  fastmap              1.1.0        2021-01-25 [1] CRAN (R 4.1.0)
#>  foreach              1.5.2        2022-02-02 [1] CRAN (R 4.1.2)
#>  fs                   1.5.2        2021-12-08 [1] CRAN (R 4.1.0)
#>  generics             0.1.3        2022-07-05 [1] CRAN (R 4.1.2)
#>  GenomeInfoDb         1.30.1       2022-01-30 [1] Bioconductor
#>  GenomeInfoDbData     1.2.7        2021-11-15 [1] Bioconductor
#>  ggplot2              3.3.6        2022-05-03 [1] CRAN (R 4.1.2)
#>  glue                 1.6.2        2022-02-24 [1] CRAN (R 4.1.2)
#>  gtable               0.3.0        2019-03-25 [1] CRAN (R 4.1.0)
#>  highr                0.9          2021-04-16 [1] CRAN (R 4.1.0)
#>  htmltools            0.5.3        2022-07-18 [1] CRAN (R 4.1.2)
#>  htmlwidgets          1.5.4        2021-09-08 [1] CRAN (R 4.1.0)
#>  httpuv               1.6.5        2022-01-05 [1] CRAN (R 4.1.2)
#>  igraph               1.3.4        2022-07-19 [1] CRAN (R 4.1.2)
#>  IRanges              2.28.0       2021-10-26 [1] Bioconductor
#>  iterators            1.0.14       2022-02-05 [1] CRAN (R 4.1.2)
#>  jsonlite             1.8.0        2022-02-22 [1] CRAN (R 4.1.2)
#>  knitr                1.40         2022-08-24 [1] CRAN (R 4.1.2)
#>  later                1.3.0        2021-08-18 [1] CRAN (R 4.1.0)
#>  lattice              0.20-45      2021-09-22 [1] CRAN (R 4.1.2)
#>  lifecycle            1.0.1        2021-09-24 [1] CRAN (R 4.1.0)
#>  lpSolveAPI           5.5.2.0-17.8 2022-08-12 [1] CRAN (R 4.1.2)
#>  magrittr             2.0.3        2022-03-30 [1] CRAN (R 4.1.2)
#>  MASS                 7.3-58.1     2022-08-03 [1] CRAN (R 4.1.2)
#>  Matrix               1.4-1        2022-03-23 [1] CRAN (R 4.1.2)
#>  memoise              2.0.1        2021-11-26 [1] CRAN (R 4.1.0)
#>  mgcv                 1.8-40       2022-03-29 [1] CRAN (R 4.1.2)
#>  mime                 0.12         2021-09-28 [1] CRAN (R 4.1.0)
#>  miniUI               0.1.1.1      2018-05-18 [1] CRAN (R 4.1.0)
#>  multtest             2.50.0       2021-10-26 [1] Bioconductor
#>  munsell              0.5.0        2018-06-12 [1] CRAN (R 4.1.0)
#>  nlme                 3.1-159      2022-08-09 [1] CRAN (R 4.1.2)
#>  numDeriv             2016.8-1.1   2019-06-06 [1] CRAN (R 4.1.0)
#>  permute              0.9-7        2022-01-27 [1] CRAN (R 4.1.2)
#>  phyloseq             1.38.0       2021-10-26 [1] Bioconductor
#>  pillar               1.8.1        2022-08-19 [1] CRAN (R 4.1.2)
#>  pkgbuild             1.3.1        2021-12-20 [1] CRAN (R 4.1.0)
#>  pkgconfig            2.0.3        2019-09-22 [1] CRAN (R 4.1.0)
#>  pkgload              1.3.0        2022-06-27 [1] CRAN (R 4.1.2)
#>  plyr                 1.8.7        2022-03-24 [1] CRAN (R 4.1.2)
#>  prettyunits          1.1.1        2020-01-24 [1] CRAN (R 4.1.0)
#>  processx             3.7.0        2022-07-07 [1] CRAN (R 4.1.2)
#>  profvis              0.3.7        2020-11-02 [1] CRAN (R 4.1.0)
#>  promises             1.2.0.1      2021-02-11 [1] CRAN (R 4.1.0)
#>  ps                   1.7.1        2022-06-18 [1] CRAN (R 4.1.2)
#>  purrr                0.3.4        2020-04-17 [1] CRAN (R 4.1.0)
#>  R.cache              0.16.0       2022-07-21 [1] CRAN (R 4.1.2)
#>  R.methodsS3          1.8.2        2022-06-13 [1] CRAN (R 4.1.2)
#>  R.oo                 1.25.0       2022-06-12 [1] CRAN (R 4.1.2)
#>  R.utils              2.12.0       2022-06-28 [1] CRAN (R 4.1.2)
#>  R6                   2.5.1        2021-08-19 [1] CRAN (R 4.1.0)
#>  Rcpp                 1.0.9        2022-07-08 [1] CRAN (R 4.1.2)
#>  RCurl                1.98-1.8     2022-07-30 [1] CRAN (R 4.1.2)
#>  registry             0.5-1        2019-03-05 [1] CRAN (R 4.1.0)
#>  remotes              2.4.2        2021-11-30 [1] CRAN (R 4.1.0)
#>  reprex               2.0.2        2022-08-17 [1] CRAN (R 4.1.2)
#>  reshape2             1.4.4        2020-04-09 [1] CRAN (R 4.1.0)
#>  rhdf5                2.38.1       2022-03-10 [1] Bioconductor
#>  rhdf5filters         1.6.0        2021-10-26 [1] Bioconductor
#>  Rhdf5lib             1.16.0       2021-10-26 [1] Bioconductor
#>  rlang                1.0.4        2022-07-12 [1] CRAN (R 4.1.2)
#>  rmarkdown            2.16         2022-08-24 [1] CRAN (R 4.1.2)
#>  ROI                  1.0-0        2020-08-31 [1] CRAN (R 4.1.0)
#>  ROI.plugin.lpsolve   1.0-1        2021-06-15 [1] CRAN (R 4.1.0)
#>  rstudioapi           0.14         2022-08-22 [1] CRAN (R 4.1.2)
#>  S4Vectors            0.32.4       2022-03-29 [1] Bioconductor
#>  scales               1.2.1        2022-08-20 [1] CRAN (R 4.1.2)
#>  sessioninfo          1.2.2        2021-12-06 [1] CRAN (R 4.1.0)
#>  shiny                1.7.2        2022-07-19 [1] CRAN (R 4.1.2)
#>  slam                 0.1-50       2022-01-08 [1] CRAN (R 4.1.2)
#>  stringi              1.7.8        2022-07-11 [1] CRAN (R 4.1.2)
#>  stringr              1.4.1        2022-08-20 [1] CRAN (R 4.1.2)
#>  styler               1.7.0        2022-03-13 [1] CRAN (R 4.1.2)
#>  survival             3.4-0        2022-08-09 [1] CRAN (R 4.1.2)
#>  tibble               3.1.8        2022-07-22 [1] CRAN (R 4.1.2)
#>  tidyselect           1.1.2        2022-02-21 [1] CRAN (R 4.1.2)
#>  urlchecker           1.0.1        2021-11-30 [1] CRAN (R 4.1.0)
#>  usethis              2.1.6        2022-05-25 [1] CRAN (R 4.1.2)
#>  utf8                 1.2.2        2021-07-24 [1] CRAN (R 4.1.0)
#>  vctrs                0.4.1        2022-04-13 [1] CRAN (R 4.1.2)
#>  vegan                2.6-2        2022-04-17 [1] CRAN (R 4.1.2)
#>  withr                2.5.0        2022-03-03 [1] CRAN (R 4.1.2)
#>  xfun                 0.32         2022-08-10 [1] CRAN (R 4.1.2)
#>  xtable               1.8-4        2019-04-21 [1] CRAN (R 4.1.0)
#>  XVector              0.34.0       2021-10-26 [1] Bioconductor
#>  yaml                 2.3.5        2022-02-21 [1] CRAN (R 4.1.2)
#>  zlibbioc             1.40.0       2021-10-26 [1] Bioconductor
#> 
#>  [1] /Library/Frameworks/R.framework/Versions/4.1/Resources/library
#> 
#> ──────────────────────────────────────────────────────────────────────────────

^{Created on 2022-08-31 with reprex v2.0.2}

[xec-cm]

The corncob dependence detectseparation has a new version for 5 days in cran (v0.3). In this new version the way to obtain the separation value is different from v0.2. Therefore these lines of code from the bbdm function fail:

https://github.com/bryandmartin/corncob/blob/7f146f2afcec703d3f21958ef55f48c4fd50a751/R/bbdml.R#L121 https://github.com/bryandmartin/corncob/blob/7f146f2afcec703d3f21958ef55f48c4fd50a751/R/bbdml.R#L129 https://github.com/bryandmartin/corncob/blob/7f146f2afcec703d3f21958ef55f48c4fd50a751/R/bbdml.R#L141 https://github.com/bryandmartin/corncob/blob/7f146f2afcec703d3f21958ef55f48c4fd50a751/R/bbdml.R#L149

In my opinion there are two solutions. The first is to indicate the version of detectseparation as v0.2 in the DESCRIPTION. The other is to adapt the code to the new version of the detectseparation package.

I hope you can resolve this issue early.

Thanks

[jorondo1]

Hi, I am having a similar problem with the vignette data:

> corncob1_da <- bbdml(formula = OTU.1 ~ DayAmdmt,
+                     phi.formula = ~ DayAmdmt,
+                     data = soil)
Error in if (detectseparation::detect_separation(y = cbind(W, M - W),  : 
  argument is of length zero

I am using the demo script that was presented at STAMPS https://raw.githubusercontent.com/statdivlab/stamps2022/main/Sunday-afternoon/labs/corncob_tutorial/corncob_tutorial.R

You seem to have fixed something yesterday, I made sure I had the most recent version of corncob but that doesn't seem to work either.

[xec-cm]

@jorondo1 I am not the maintainer of this repository... My workaround (as long as Corncob is not updated) was to force installion of the version 0.2 of the detectseparation package. You can do this easily with devtools or remotes:

## Using remotes
remotes::install_version("detectseparation", version = 0.2)

## Using devtools
devtools::install_version("detectseparation", version = 0.2)

After installation, you just need to restart R and run the code normally.

[adw96]

Thanks for posting this issue and your workaround @xec-cm, and for corroborating it, @jorondo1 ! Please note that the software authors have taken note of this issue and are working to come up with a long term solution soon. We will post updates here.

cc @bryandmartin

[jorondo1]

Thank you both, the workaround fixed it for me as well. Cheers !

[bryandmartin]

Fixed in 365add6