search

perishky / meffil

Efficient algorithms for analyzing DNA methylation data.
Artistic License 2.0
55 stars 28 forks source link

dimnames error in meffil.normalize.samples() #33

Closed tomkuipers1402 closed 3 years ago

tomkuipers1402 commented 3 years ago

Hi,

I'm trying to run meffil.normalize.samples() on my data, but it keeps returning me this error:

Error in dimnames(ret) <- list(sites, names(norm.objects)) :
  length of 'dimnames' [1] not equal to array extent
In addition: Warning message:
In mclapply(X, FUN, ...) :
  all scheduled cores encountered errors in user code

I then tried your steps in readme.md, but this gave the same error as above. After downloading and extracting the test data, I used this code:

samplesheet <- meffil.create.samplesheet(path)
qc.objects <- meffil.qc(samplesheet, verbose = TRUE)

qc.summary <- meffil.qc.summary(qc.objects)
meffil.qc.report(qc.summary, output.file="qc_test/report.html")

qc.objects <- meffil.remove.samples(qc.objects, qc.summary$bad.samples$sample.name)
norm.objects <- meffil.normalize.quantiles(qc.objects, number.pcs=10)

norm.beta <- meffil.normalize.samples(norm.objects, cpglist.remove=qc.summary$bad.cpgs$name, verbose = TRUE)

Any idea what's going wrong? Best regards, Tom

R version 4.0.3 (2020-10-10)
Platform: x86_64-conda-linux-gnu (64-bit)
Running under: CentOS Stream 8

Matrix products: default
BLAS/LAPACK: /tbkuipers/miniconda3/envs/R/lib/libopenblasp-r0.3.12.so

locale:
 [1] LC_CTYPE=C.UTF-8       LC_NUMERIC=C           LC_TIME=C.UTF-8
 [4] LC_COLLATE=C.UTF-8     LC_MONETARY=C.UTF-8    LC_MESSAGES=C.UTF-8
 [7] LC_PAPER=C.UTF-8       LC_NAME=C              LC_ADDRESS=C
[10] LC_TELEPHONE=C         LC_MEASUREMENT=C.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] parallel  stats     graphics  grDevices utils     datasets  methods
[8] base

other attached packages:
 [1] meffil_1.1.2          preprocessCore_1.52.1 SmartSVA_0.1.3
 [4] RSpectra_0.16-0       isva_1.9              JADE_2.0-3
 [7] qvalue_2.22.0         gdsfmt_1.26.1         statmod_1.4.35
[10] quadprog_1.5-8        DNAcopy_1.64.0        fastICA_1.2-2
[13] lme4_1.1-26           Matrix_1.3-2          multcomp_1.4-17
[16] TH.data_1.0-10        survival_3.2-7        mvtnorm_1.1-2
[19] matrixStats_0.57.0    markdown_1.1          gridExtra_2.3
[22] Cairo_1.5-12.2        knitr_1.30            reshape2_1.4.4
[25] plyr_1.8.6            ggplot2_3.3.3         sva_3.38.0
[28] BiocParallel_1.24.1   genefilter_1.72.1     mgcv_1.8-33
[31] nlme_3.1-151          limma_3.46.0          MASS_7.3-53
[34] illuminaio_0.32.0

loaded via a namespace (and not attached):
 [1] bit64_4.0.5          httr_1.4.2           tools_4.0.3
 [4] R6_2.5.0             DBI_1.1.1            BiocGenerics_0.36.0
 [7] colorspace_2.0-0     withr_2.4.1          tidyselect_1.1.0
[10] base64_2.0           bit_4.0.4            compiler_4.0.3
[13] Biobase_2.50.0       sandwich_3.0-1       labeling_0.4.2
[16] scales_1.1.1         askpass_1.1          stringr_1.4.0
[19] digest_0.6.27        minqa_1.2.4          pkgconfig_2.0.3
[22] highr_0.8            fastmap_1.0.1        rlang_0.4.10
[25] RSQLite_2.2.2        generics_0.1.0       farver_2.0.3
[28] zoo_1.8-8            dplyr_1.0.3          magrittr_2.0.1
[31] Rcpp_1.0.6           munsell_0.5.0        S4Vectors_0.28.1
[34] lifecycle_1.0.0      stringi_1.5.3        edgeR_3.32.1
[37] grid_4.0.3           blob_1.2.1           crayon_1.3.4
[40] lattice_0.20-41      splines_4.0.3        annotate_1.68.0
[43] locfit_1.5-9.4       pillar_1.4.7         boot_1.3-25
[46] codetools_0.2-18     stats4_4.0.3         XML_3.99-0.5
[49] glue_1.4.2           evaluate_0.14        vctrs_0.3.6
[52] nloptr_1.2.2.2       gtable_0.3.0         openssl_1.4.3
[55] purrr_0.3.4          clue_0.3-59          assertthat_0.2.1
[58] cachem_1.0.4         xfun_0.20            mime_0.9
[61] xtable_1.8-4         tibble_3.0.5         AnnotationDbi_1.52.0
[64] memoise_2.0.0        IRanges_2.24.1       cluster_2.1.0
[67] ellipsis_0.3.1
perishky commented 3 years ago

Hi Tom,

In this case, the warning message is providing the most information: "mclapply: all scheduled cores encountered errors in user code"

It suggests to me that something is going wrong with individual samples. Unfortunately mclapply which is being used to parallelize normalization doesn't pass on error information for individual samples.

Could you try normalizing a few individual samples to obtain more informative error messages, e.g.

ret <- meffil.normalize.sample(norm.objects[[1]]) ret <- meffil.normalize.sample(norm.objects[[2]]) ...

When an error is generated, please send the error message on to me. It would be also helpful to send the output of the following command immediately after the error is generated: traceback()

Best, Matt

From: Tom Kuipers @.> Sent: 24 June 2021 12:10 PM To: perishky/meffil @.> Cc: Subscribed @.***> Subject: [perishky/meffil] dimnames error in meffil.normalize.samples() (#33)

Hi,

I'm trying to run meffil.normalize.samples() on my data, but it keeps returning me this error:

Error in dimnames(ret) <- list(sites, names(norm.objects)) : length of 'dimnames' [1] not equal to array extent In addition: Warning message: In mclapply(X, FUN, ...) : all scheduled cores encountered errors in user code

I then tried your steps in readme.md, but this gave the same error as above. After downloading and extracting the test data, I used this code:

samplesheet <- meffil.create.samplesheet(path) qc.objects <- meffil.qc(samplesheet, verbose = TRUE)

qc.summary <- meffil.qc.summary(qc.objects) meffil.qc.report(qc.summary, output.file="qc_test/report.html")

qc.objects <- meffil.remove.samples(qc.objects, qc.summary$bad.samples$sample.name) norm.objects <- meffil.normalize.quantiles(qc.objects, number.pcs=10)

norm.beta <- meffil.normalize.samples(norm.objects, cpglist.remove=qc.summary$bad.cpgs$name, verbose = TRUE)

Any idea what's going wrong? Best regards, Tom

R version 4.0.3 (2020-10-10) Platform: x86_64-conda-linux-gnu (64-bit) Running under: CentOS Stream 8

Matrix products: default BLAS/LAPACK: /tbkuipers/miniconda3/envs/R/lib/libopenblasp-r0.3.12.so

locale: [1] LC_CTYPE=C.UTF-8 LC_NUMERIC=C LC_TIME=C.UTF-8 [4] LC_COLLATE=C.UTF-8 LC_MONETARY=C.UTF-8 LC_MESSAGES=C.UTF-8 [7] LC_PAPER=C.UTF-8 LC_NAME=C LC_ADDRESS=C [10] LC_TELEPHONE=C LC_MEASUREMENT=C.UTF-8 LC_IDENTIFICATION=C

attached base packages: [1] parallel stats graphics grDevices utils datasets methods [8] base

other attached packages: [1] meffil_1.1.2 preprocessCore_1.52.1 SmartSVA_0.1.3 [4] RSpectra_0.16-0 isva_1.9 JADE_2.0-3 [7] qvalue_2.22.0 gdsfmt_1.26.1 statmod_1.4.35 [10] quadprog_1.5-8 DNAcopy_1.64.0 fastICA_1.2-2 [13] lme4_1.1-26 Matrix_1.3-2 multcomp_1.4-17 [16] TH.data_1.0-10 survival_3.2-7 mvtnorm_1.1-2 [19] matrixStats_0.57.0 markdown_1.1 gridExtra_2.3 [22] Cairo_1.5-12.2 knitr_1.30 reshape2_1.4.4 [25] plyr_1.8.6 ggplot2_3.3.3 sva_3.38.0 [28] BiocParallel_1.24.1 genefilter_1.72.1 mgcv_1.8-33 [31] nlme_3.1-151 limma_3.46.0 MASS_7.3-53 [34] illuminaio_0.32.0

loaded via a namespace (and not attached): [1] bit64_4.0.5 httr_1.4.2 tools_4.0.3 [4] R6_2.5.0 DBI_1.1.1 BiocGenerics_0.36.0 [7] colorspace_2.0-0 withr_2.4.1 tidyselect_1.1.0 [10] base64_2.0 bit_4.0.4 compiler_4.0.3 [13] Biobase_2.50.0 sandwich_3.0-1 labeling_0.4.2 [16] scales_1.1.1 askpass_1.1 stringr_1.4.0 [19] digest_0.6.27 minqa_1.2.4 pkgconfig_2.0.3 [22] highr_0.8 fastmap_1.0.1 rlang_0.4.10 [25] RSQLite_2.2.2 generics_0.1.0 farver_2.0.3 [28] zoo_1.8-8 dplyr_1.0.3 magrittr_2.0.1 [31] Rcpp_1.0.6 munsell_0.5.0 S4Vectors_0.28.1 [34] lifecycle_1.0.0 stringi_1.5.3 edgeR_3.32.1 [37] grid_4.0.3 blob_1.2.1 crayon_1.3.4 [40] lattice_0.20-41 splines_4.0.3 annotate_1.68.0 [43] locfit_1.5-9.4 pillar_1.4.7 boot_1.3-25 [46] codetools_0.2-18 stats4_4.0.3 XML_3.99-0.5 [49] glue_1.4.2 evaluate_0.14 vctrs_0.3.6 [52] nloptr_1.2.2.2 gtable_0.3.0 openssl_1.4.3 [55] purrr_0.3.4 clue_0.3-59 assertthat_0.2.1 [58] cachem_1.0.4 xfun_0.20 mime_0.9 [61] xtable_1.8-4 tibble_3.0.5 AnnotationDbi_1.52.0 [64] memoise_2.0.0 IRanges_2.24.1 cluster_2.1.0 [67] ellipsis_0.3.1

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/perishky/meffil/issues/33, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ACSRUUPZXD6HPQ5IGJYJNSTTUMHBVANCNFSM47HVFYKQ.

tomkuipers1402 commented 3 years ago

Hi Matt,

Thanks for the response, I'm getting these errors now:

> ret <- meffil.normalize.sample(norm.objects[[1]])
Error in preprocessCore::normalize.quantiles.use.target(matrix(orig.signal),  :
  ERROR; return code from pthread_create() is 22
In addition: Warning messages:
1: In readChar(con, nchars = n) : truncating string with embedded nuls
2: In readChar(con, nchars = n) : truncating string with embedded nuls
> traceback()
2: preprocessCore::normalize.quantiles.use.target(matrix(orig.signal),
       norm.target)
1: meffil.normalize.sample(norm.objects[[1]])
perishky commented 3 years ago

Hi Tom,

This looks to me like there might be something wrong/unexpected about the IDAT file. Do you get this same error for other files as well?

Best, Matt

From: Tom Kuipers @.> Sent: 25 June 2021 8:57 AM To: perishky/meffil @.> Cc: Matthew Suderman @.>; Comment @.> Subject: Re: [perishky/meffil] dimnames error in meffil.normalize.samples() (#33)

Hi Matt,

Thanks for the response, I'm getting these errors now:

ret <- meffil.normalize.sample(norm.objects[[1]]) Error in preprocessCore::normalize.quantiles.use.target(matrix(orig.signal), : ERROR; return code from pthread_create() is 22 In addition: Warning messages: 1: In readChar(con, nchars = n) : truncating string with embedded nuls 2: In readChar(con, nchars = n) : truncating string with embedded nuls traceback() 2: preprocessCore::normalize.quantiles.use.target(matrix(orig.signal), norm.target) 1: meffil.normalize.sample(norm.objects[[1]])

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/perishky/meffil/issues/33#issuecomment-868301567, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ACSRUUNOXKUBEJA7R7DTTVTTUQZHJANCNFSM47HVFYKQ.

tomkuipers1402 commented 3 years ago

Hi Matt,

I've just solved the issue. It was related to the "preprocessCore" package, and was fixed by first removing & reinstalling preprocessCore:

remove.packages(pkgs = "preprocessCore")
BiocManager::install("preprocessCore", configure.args="--disable-threading")

Thanks for your help! Best Tom

perishky commented 3 years ago

Great work. I'll keep this in mind if I see this error come up again.

From: Tom Kuipers @.> Sent: 29 June 2021 3:55 PM To: perishky/meffil @.> Cc: Matthew Suderman @.>; Comment @.> Subject: Re: [perishky/meffil] dimnames error in meffil.normalize.samples() (#33)

Hi Matt,

I've just solved the issue. It was related to the "preprocessCore" package, and was fixed by first removing & reinstalling preprocessCore:

remove.packages(pkgs = "preprocessCore") BiocManager::install("preprocessCore", configure.args="--disable-threading")

Thanks for your help! Best Tom

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/perishky/meffil/issues/33#issuecomment-870672974, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ACSRUUIFQZYSDH5Y3UHYKHDTVHNHPANCNFSM47HVFYKQ.