learnErrors, dada and removeBimeraDenovo result in ‘caught segfault’ error

[petrijp]

Hi,

learnErrors and dada with multithread = TRUE and removeBimeraDenovo result in ‘caught segfault’ error, please see the details below. With default multithread = FALSE learnErrors and dada work as expected, whereas removeBimeraDenovo gives the same error with multithread = FALSE, too. R version is the latest and dada2 and the dependencies are freshly installed. I’d really appreciate any suggestions on how to get rid of the problem!

Details: R version 4.2.2 (2022-10-31) Platform: aarch64-apple-darwin20 (64-bit) Running under: macOS Ventura 13.0.1 dada2 1.26.0

errF <- learnErrors(filtFs[c(1,3)], multithread=TRUE)

9041529 total bases in 61507 reads from 2 samples will be used for learning the error rates. caught segfault address 0x9ffffffe7, cause 'invalid permissions' Traceback: 1: dada_uniques(names(drpi$uniques), unname(drpi$uniques), names(drpi$uniques) %in% c(priors, pseudo_priors), erri, unname(t(drpi$quals)), opts[["MATCH"]], opts[["MISMATCH"]], opts[["GAP_PENALTY"]], opts[["USE_KMERS"]], opts[["KDIST_CUTOFF"]], opts[["BAND_SIZE"]], opts[["OMEGA_A"]], opts[["OMEGA_P"]], opts[["OMEGA_C"]], opts[["DETECT_SINGLETONS"]], if (initializeErr) { 1 } else { opts[["MAX_CLUST"]] }, opts[["MIN_FOLD"]], opts[["MIN_HAMMING"]], opts[["MIN_ABUNDANCE"]], TRUE, FALSE, opts[["VECTORIZED_ALIGNMENT"]], opts[["HOMOPOLYMER_GAP_PENALTY"]], multithread, (verbose >= 2), opts[["SSE"]], opts[["GAPLESS"]], opts[["GREEDY"]]) 2: dada(drps, err = NULL, errorEstimationFunction = errorEstimationFunction, selfConsist = TRUE, multithread = multithread, verbose = verbose, MAX_CONSIST = MAX_CONSIST, OMEGA_C = OMEGA_C, ...) 3: learnErrors(filtFs[c(1, 3)], multithread = TRUE)

dadaFs <- dada(derepFs, err=errF, multithread=TRUE)

caught segfault address 0x9ffffffe7, cause 'invalid permissions' Traceback: 1: dada_uniques(names(drpi$uniques), unname(drpi$uniques), names(drpi$uniques) %in% c(priors, pseudo_priors), erri, unname(t(drpi$quals)), opts[["MATCH"]], opts[["MISMATCH"]], opts[["GAP_PENALTY"]], opts[["USE_KMERS"]], opts[["KDIST_CUTOFF"]], opts[["BAND_SIZE"]], opts[["OMEGA_A"]], opts[["OMEGA_P"]], opts[["OMEGA_C"]], opts[["DETECT_SINGLETONS"]], if (initializeErr) { 1 } else { opts[["MAX_CLUST"]] }, opts[["MIN_FOLD"]], opts[["MIN_HAMMING"]], opts[["MIN_ABUNDANCE"]], TRUE, FALSE, opts[["VECTORIZED_ALIGNMENT"]], opts[["HOMOPOLYMER_GAP_PENALTY"]], multithread, (verbose >= 2), opts[["SSE"]], opts[["GAPLESS"]], opts[["GREEDY"]]) 2: dada(derepFs, err = errF, multithread = TRUE)

seqtab.nochim <- removeBimeraDenovo(seqtab, method="consensus", multithread=TRUE, verbose=TRUE)

caught segfault address 0x9ffffffe7, cause 'invalid permissions' Traceback: 1: C_table_bimera2(seqtab, sqs, minFoldParentOverAbundance, minParentAbundance, allowOneOff, minOneOffParentDistance, getDadaOpt("MATCH"), getDadaOpt("MISMATCH"), getDadaOpt("GAP_PENALTY"), maxShift) 2: isBimeraDenovoTable(unqs[[i]], ..., verbose = verbose) 3: removeBimeraDenovo(seqtab, method = "consensus", multithread = TRUE, verbose = TRUE)

seqtab.nochim <- removeBimeraDenovo(seqtab, method="consensus", multithread=FALSE, verbose=TRUE)

caught segfault address 0x9ffffffe7, cause 'invalid permissions' Traceback: 1: C_table_bimera2(seqtab, sqs, minFoldParentOverAbundance, minParentAbundance, allowOneOff, minOneOffParentDistance, getDadaOpt("MATCH"), getDadaOpt("MISMATCH"), getDadaOpt("GAP_PENALTY"), maxShift) 2: isBimeraDenovoTable(unqs[[i]], ..., verbose = verbose) 3: removeBimeraDenovo(seqtab, method = "consensus", multithread = FALSE, verbose = TRUE)

[benjjneb]

At first glance, I have no idea? I'm not sure I've ever seen a similar error report, where a segfault went away in all but removeBimeraDenovo when turning off multithreading.

I am on the same platform (R/dada2/aarch64-apple-darwin20), but one behind in the OS (Monterey 12.0.1) and do not see this issue on my machine.

I hate to go straight to this, but does it persist with a full on reinstall of R and the packages? Also, our multithreading is mostly leveraging the RcppParallel package, what versions of Rcpp and RcppParallel do you have¿

[marclliros]

Hello Similar issue happened to me after upgrading to Ventura(13.0.1 (22A400)). I have reinstalled R and Rcpp and RcppParallel too. This is the sessionInfo() output

sessionInfo() R version 4.2.2 (2022-10-31) Platform: x86_64-apple-darwin17.0 (64-bit) Running under: macOS Ventura 13.0.1

Matrix products: default BLAS: /Library/Frameworks/R.framework/Versions/4.2/Resources/lib/libRblas.0.dylib LAPACK: /Library/Frameworks/R.framework/Versions/4.2/Resources/lib/libRlapack.dylib

locale: [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages: [1] stats graphics grDevices utils datasets methods base

loaded via a namespace (and not attached): [1] tidyselect_1.2.0 reshape2_1.4.4 splines_4.2.2 lattice_0.20-45
[5] rhdf5_2.42.0 colorspace_2.0-3 vctrs_0.5.1 generics_0.1.3
[9] stats4_4.2.2 mgcv_1.8-41 survival_3.4-0 utf8_1.2.2
[13] rlang_1.0.6 pillar_1.8.1 glue_1.6.2 DBI_1.1.3
[17] BiocGenerics_0.44.0 GenomeInfoDbData_1.2.9 foreach_1.5.2 lifecycle_1.0.3
[21] plyr_1.8.8 stringr_1.4.1 zlibbioc_1.44.0 Biostrings_2.66.0
[25] munsell_0.5.0 gtable_0.3.1 codetools_0.2-18 phyloseq_1.42.0
[29] Biobase_2.58.0 permute_0.9-7 IRanges_2.32.0 GenomeInfoDb_1.34.3
[33] biomformat_1.26.0 parallel_4.2.2 fansi_1.0.3 Rcpp_1.0.9
[37] scales_1.2.1 vegan_2.6-4 S4Vectors_0.36.0 jsonlite_1.8.3
[41] XVector_0.38.0 ggplot2_3.4.0 stringi_1.7.8 dplyr_1.0.10
[45] grid_4.2.2 ade4_1.7-20 cli_3.4.1 tools_4.2.2
[49] bitops_1.0-7 rhdf5filters_1.10.0 magrittr_2.0.3 RCurl_1.98-1.9
[53] tibble_3.1.8 cluster_2.1.4 crayon_1.5.2 ape_5.6-2
[57] pkgconfig_2.0.3 MASS_7.3-58.1 Matrix_1.5-3 data.table_1.14.6
[61] assertthat_0.2.1 iterators_1.0.14 Rhdf5lib_1.20.0 R6_2.5.1
[65] multtest_2.54.0 igraph_1.3.5 nlme_3.1-160 compiler_4.2.2

I have not been able to go ahead after ...

errF<-learnErrors(filtFs,multithread=TRUE) 110783520 total bases in 461598 reads from 3 samples will be used for learning the error rates.

caught segfault address 0x9ffffffe7, cause 'memory not mapped'

Traceback: 1: dada_uniques(names(drpi$uniques), unname(drpi$uniques), names(drpi$uniques) %in% c(priors, pseudo_priors), erri, unname(t(drpi$quals)), opts[["MATCH"]], opts[["MISMATCH"]], opts[["GAP_PENALTY"]], opts[["USE_KMERS"]], opts[["KDIST_CUTOFF"]], opts[["BAND_SIZE"]], opts[["OMEGA_A"]], opts[["OMEGA_P"]], opts[["OMEGA_C"]], opts[["DETECT_SINGLETONS"]], if (initializeErr) { 1 } else { opts[["MAX_CLUST"]] }, opts[["MIN_FOLD"]], opts[["MIN_HAMMING"]], opts[["MIN_ABUNDANCE"]], TRUE, FALSE, opts[["VECTORIZED_ALIGNMENT"]], opts[["HOMOPOLYMER_GAP_PENALTY"]], multithread, (verbose >= 2), opts[["SSE"]], opts[["GAPLESS"]], opts[["GREEDY"]]) 2: dada(drps, err = NULL, errorEstimationFunction = errorEstimationFunction, selfConsist = TRUE, multithread = multithread, verbose = verbose, MAX_CONSIST = MAX_CONSIST, OMEGA_C = OMEGA_C, ...) 3: learnErrors(filtFs, multithread = TRUE)

Possible actions: 1: abort (with core dump, if enabled) 2: normal R exit 3: exit R without saving workspace 4: exit R saving workspace Selection:

Any idea?? Thanks for al the efforts done on heklping us @benjjneb

Yours Marc

[benjjneb]

As of now this remains not understood. The two examples in this thread had errors using different function calls in the dada2 R package, and with different segfault error messages.

In the absence of further reports, this will remain as is. I can't reproduce these segfaults on my machine at least.

If in the future a reproducible example could be generated, that could get a real investigation started.

[kcairns]

@benjjneb I have a macos running Ventura 13.4.1 (22F82) and am getting the same seg fault.

Newly updated R and installed dada2 via bioconductor.

I am getting the same segmentation fault as marclliros when I try to run learnErrors with Multithreading, I managed to work around it by setting multithread=FALSE but that isn't fixing the BimeraDenovo issue where I get a segfault.

seqtab.nochim <- removeBimeraDenovo(seqtab, method="consensus", multithread=FALSE, verbose=TRUE) or with multithread=TRUE. I also had segmentation faults for

caught segfault address 0x9ffffffe7, cause 'memory not mapped'

Traceback: 1: C_table_bimera2(seqtab, sqs, minFoldParentOverAbundance, minParentAbundance, allowOneOff, minOneOffParentDistance, getDadaOpt("MATCH"), getDadaOpt("MISMATCH"), getDadaOpt("GAP_PENALTY"), maxShift) 2: isBimeraDenovoTable(unqs[[i]], ..., verbose = verbose) 3: removeBimeraDenovo(seqtab, method = "consensus", multithread = TRUE, verbose = TRUE)

Has anyone else experienced this and had a fix?

I am wondering if it is a Ventura issue, and/or the new R version.

[benjjneb]

Newly updated R and installed dada2 via bioconductor.

Can you specify the R version and package version numbers?

Has anyone else experienced this and had a fix?

I still have not been able to replicate, so no fix on my end. Are you able to compile packages yourself? If so, it might be worth exploring that... I wonder if there is something going on with x86 compilation -> rosetta translation for Mac M1/2 architectures?

[jorgeibarracaballero]

Hello. I am having the same problem, running R version 4.2.1, dada2 1.26.0. Apparently it started after a MacOs Ventura update (or after updating to Ventura, maybe, not sure), because previously it was working in the same machine with the same R and dada2 versions (and dependencies, in Mac mini INTEL). Then I installed R and dada2 in a new MacBook Air M2, and have the same problem: seqtab.nochim even with multithread=FALSE produces the "segfault". I tried to find a solution online but I only find old and confusing answers. I hope this can be solved, now I can run it only in an old iMac. Thank you

[kcairns]

My R version is 4.3.1 and the DADA2 version is 1.26.0 - dependencies are installed. I did a clean install of R and DADA2, and had tried using R 4.2.1 before updating it all.

My Mac is an intel based Macbook Pro.

[murheisa]

Hello, I´m having the same issue on a MacBook Air m1 8gb. I get as well this:

caught segfault

and asks me to exit when I use errF<-learnErrors(filtFs,multithread=TRUE), if I change multithread=FALSE, I can continue, but by the time I get to RemoveBimeraDenovo, even with multithread=FALSE the app stops responding, and I must exit and start all over again. I didn't have the chance to copy the rest of caugh segfault because it crashed and had to force exit.

I'm working with 18 samples, dada version is 1.28.0 and R 4.3.1, Ventura 13.3

[petrijp]

Hei, the problem persists in MacBook Pro M1 Max, macOS14 with newly installed dada2 and updated dependencies. I have the after-crash error report, I wonder if it tells the reason for those who understand how computers work… It’s 29 pages in Calibri font, size 12, perhaps too long to be posted here?

sessionInfo() R version 4.3.1 (2023-06-16) Platform: aarch64-apple-darwin20 (64-bit) Running under: macOS Sonoma 14.0

Matrix products: default BLAS: /Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/lib/libRblas.0.dylib LAPACK: /Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/lib/libRlapack.dylib; LAPACK version 3.11.0

locale: [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

time zone: Asia/Shanghai tzcode source: internal

attached base packages: [1] stats graphics grDevices utils datasets methods base

other attached packages: [1] dada2_1.28.0 Rcpp_1.0.11

loaded via a namespace (and not attached): [1] utf8_1.2.4 generics_0.1.3 bitops_1.0-7
[4] stringi_1.7.12 jpeg_0.1-10 lattice_0.22-5
[7] magrittr_2.0.3 grid_4.3.1 RColorBrewer_1.1-3
[10] plyr_1.8.9 Matrix_1.6-1.1 GenomeInfoDb_1.36.4
[13] fansi_1.0.5 scales_1.2.1 Biostrings_2.68.1
[16] codetools_0.2-19 abind_1.4-5 cli_3.6.1
[19] ShortRead_1.58.0 rlang_1.1.1 crayon_1.5.2
[22] XVector_0.40.0 Biobase_2.60.0 munsell_0.5.0
[25] DelayedArray_0.26.7 S4Arrays_1.0.6 tools_4.3.1
[28] parallel_4.3.1 reshape2_1.4.4 deldir_1.0-9
[31] BiocParallel_1.34.2 dplyr_1.1.3 interp_1.1-4
[34] colorspace_2.1-0 ggplot2_3.4.4 GenomeInfoDbData_1.2.10
[37] Rsamtools_2.16.0 hwriter_1.3.2.1 SummarizedExperiment_1.30.2 [40] BiocGenerics_0.46.0 png_0.1-8 vctrs_0.6.4
[43] R6_2.5.1 matrixStats_1.0.0 stats4_4.3.1
[46] lifecycle_1.0.3 stringr_1.5.0 zlibbioc_1.46.0
[49] S4Vectors_0.38.2 IRanges_2.34.1 pkgconfig_2.0.3
[52] RcppParallel_5.1.7 pillar_1.9.0 gtable_0.3.4
[55] glue_1.6.2 tibble_3.2.1 GenomicAlignments_1.36.0
[58] GenomicRanges_1.52.1 tidyselect_1.2.0 MatrixGenerics_1.12.3
[61] latticeExtra_0.6-30 compiler_4.3.1 RCurl_1.98-1.13

multithread=TRUE results in an error

errF <- learnErrors(filtFs, multithread=TRUE) 41032000 total bases in 164128 reads from 2 samples will be used for learning the error rates.

caught segfault address 0x9ffffffe7, cause 'invalid permissions'

Traceback: 1: dada_uniques(names(drpi$uniques), unname(drpi$uniques), names(drpi$uniques) %in% c(priors, pseudo_priors), erri, unname(t(drpi$quals)), opts[["MATCH"]], opts[["MISMATCH"]], opts[["GAP_PENALTY"]], opts[["USE_KMERS"]], opts[["KDIST_CUTOFF"]], opts[["BAND_SIZE"]], opts[["OMEGA_A"]], opts[["OMEGA_P"]], opts[["OMEGA_C"]], opts[["DETECT_SINGLETONS"]], if (initializeErr) { 1 } else { opts[["MAX_CLUST"]] }, opts[["MIN_FOLD"]], opts[["MIN_HAMMING"]], opts[["MIN_ABUNDANCE"]], TRUE, FALSE, opts[["VECTORIZED_ALIGNMENT"]], opts[["HOMOPOLYMER_GAP_PENALTY"]], multithread, (verbose >= 2), opts[["SSE"]], opts[["GAPLESS"]], opts[["GREEDY"]]) 2: dada(drps, err = NULL, errorEstimationFunction = errorEstimationFunction, selfConsist = TRUE, multithread = multithread, verbose = verbose, MAX_CONSIST = MAX_CONSIST, OMEGA_C = OMEGA_C, ...) 3: learnErrors(filtFs, multithread = TRUE)

Possible actions: 1: abort (with core dump, if enabled) 2: normal R exit 3: exit R without saving workspace 4: exit R saving workspace

using multithread=FALSE works OK till seqtab.nochim

seqtab.nochim <- removeBimeraDenovo(seqtab, method="consensus", multithread=FALSE, verbose=TRUE)

caught segfault address 0x9ffffffe7, cause 'invalid permissions'

Traceback: 1: C_table_bimera2(seqtab, sqs, minFoldParentOverAbundance, minParentAbundance, allowOneOff, minOneOffParentDistance, getDadaOpt("MATCH"), getDadaOpt("MISMATCH"), getDadaOpt("GAP_PENALTY"), maxShift) 2: isBimeraDenovoTable(unqs[[i]], ..., verbose = verbose) 3: removeBimeraDenovo(seqtab, method = "consensus", multithread = FALSE, verbose = TRUE)

[benjjneb]

Any outside help or suggestions on this would be much appreciated.

I am still stuck doing anything because I can't reproduce this, and I'm not familiar enough with segfault error messages to divine what the cause might be from them.

[jorgeibarracaballero]

It is not the processor in the Mac computers, it is a problem of the OS starting with Ventura. It runs fine in older Mac not updated to Ventura. It also works in an Ubuntu virtual machine

[murheisa]

Does anyone have been able to work with dada2 in Ventura or Sonoma in march 2024?? I´ve been struggling a lot with the same error with multithread

[benjjneb]

I have never been able to reproduce this issue, but I also have been getting additional reports of issues with new OSX versions. The package builds and runs simple tests using multithread=2 on the Biodonductor build system. But that level of testing may not be sufficient.

[kegarrison]

Hi! I am running a MacBook Pro with the Apple M2 chip. I have Ventura 13.6.6 installed. I have done a recent install of dada2 via Bioconductor. I am working with the tutorial dataset. My dada2 version is 1.30.0 I am getting the same errors with respect to multithreading. I can allow it in the filter/trim step without causing any problems. I need to set it to FALSE during the error rate learning and dadaFs/dadaRs step to avoid the error caught segfault address 0x9ffffffe7, cause 'invalid permissions' However, this error appears later during the seqtab.nochim step. I thought it might be because I had the MiSeq_SOP inside of a sub-directory, because R asked for permission to access my Documents folder during one of the earlier steps. However, the error occurs even when I move MiSeq_SOP to the top level of my user directory. I was not sure if this info would be helpful for troubleshooting, but I thought I should share it just in case.

Note added later: The segfault error appears to go away when executing the seqtab.nochim step without multithreading from RStudio instead of the R command line interface.

Thanks! Keith

[benjjneb]

I still cannot reproduce this bug.

On an Apple M1Pro machine (2021 Powerbook laptop) with Sonoma 14.2.1 installed, I can run the tutorial workflow with multithread=TRUE at each relevant step on the tutorial dataset or the expanded version of that dataset without error.

Ideas on what is going on here and how I could potentially reproduce it are welcome.