search

HelenaLC / CATALYST

Cytometry dATa anALYsis Tools
67 stars 31 forks source link

plotDiffHeatmap error #164

Closed carl-coyle closed 3 years ago

carl-coyle commented 3 years ago

Hi there, I've ran Differential abundance analysis on my data and have managed to pull out things like top table etc however when I try and run the plotDiffHeatmap it turns up the following error.

Screen Shot 2020-11-24 at 12 23 28

I am quite new to R so not too sure what this error in referring to. Apologies if this also doesn't give as much information needed to address a solution id be happy to provide more information which is deemed necessary.

Session Info. R version 4.0.3 (2020-10-10) Platform: x86_64-apple-darwin17.0 (64-bit) Running under: macOS High Sierra 10.13.6

**Matrix products: default BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib LAPACK: /Library/Frameworks/R.framework/Versions/4.0/Resources/lib/libRlapack.dylib

locale: [1] en_GB.UTF-8/en_GB.UTF-8/en_GB.UTF-8/C/en_GB.UTF-8/en_GB.UTF-8

attached base packages: [1] parallel stats4 stats graphics grDevices utils datasets methods base

other attached packages: [1] lme4_1.1-25 Matrix_1.2-18 diffcyt_1.10.0 cowplot_1.1.0 CATALYST_1.14.0
[6] SingleCellExperiment_1.12.0 SummarizedExperiment_1.20.0 Biobase_2.50.0 GenomicRanges_1.42.0 GenomeInfoDb_1.26.0
[11] IRanges_2.24.0 S4Vectors_0.28.0 BiocGenerics_0.36.0 MatrixGenerics_1.2.0 matrixStats_0.57.0
[16] readxl_1.3.1 forcats_0.5.0 stringr_1.4.0 dplyr_1.0.2 purrr_0.3.4
[21] readr_1.4.0 tidyr_1.1.2 tibble_3.0.4 ggplot2_3.3.2 tidyverse_1.3.0
[26] CytoML_2.2.1 flowWorkspace_4.2.0 flowCore_2.2.0

loaded via a namespace (and not attached): [1] backports_1.2.0 circlize_0.4.11 drc_3.0-1 plyr_1.8.6 igraph_1.2.6
[6] ConsensusClusterPlus_1.54.0 splines_4.0.3 BiocParallel_1.24.1 scater_1.18.3 TH.data_1.0-10
[11] digest_0.6.27 viridis_0.5.1 fansi_0.4.1 magrittr_2.0.1 cluster_2.1.0
[16] limma_3.46.0 aws.signature_0.6.0 openxlsx_4.2.3 ComplexHeatmap_2.6.2 modelr_0.1.8
[21] RcppParallel_5.0.2 sandwich_3.0-0 cytolib_2.2.0 jpeg_0.1-8.1 colorspace_2.0-0
[26] rvest_0.3.6 ggrepel_0.8.2 haven_2.3.1 xfun_0.19 crayon_1.3.4
[31] RCurl_1.98-1.2 jsonlite_1.7.1 hexbin_1.28.1 graph_1.68.0 survival_3.2-7
[36] zoo_1.8-8 glue_1.4.2 gtable_0.3.0 nnls_1.4 zlibbioc_1.36.0
[41] XVector_0.30.0 GetoptLong_1.0.4 DelayedArray_0.16.0 ggcyto_1.18.0 BiocSingular_1.6.0
[46] car_3.0-10 Rgraphviz_2.34.0 shape_1.4.5 abind_1.4-5 scales_1.1.1
[51] mvtnorm_1.1-1 edgeR_3.32.0 DBI_1.1.0 Rcpp_1.0.5 plotrix_3.7-8
[56] viridisLite_0.3.0 clue_0.3-57 rsvd_1.0.3 foreign_0.8-80 FlowSOM_1.22.0
[61] tsne_0.1-3 httr_1.4.2 RColorBrewer_1.1-2 ellipsis_0.3.1 farver_2.0.3
[66] pkgconfig_2.0.3 XML_3.99-0.5 scuttle_1.0.0 uwot_0.1.9 dbplyr_2.0.0
[71] locfit_1.5-9.4 labeling_0.4.2 tidyselect_1.1.0 rlang_0.4.8 reshape2_1.4.4
[76] munsell_0.5.0 cellranger_1.1.0 tools_4.0.3 cli_2.1.0 generics_0.1.0
[81] broom_0.7.2 ggridges_0.5.2 aws.s3_0.3.21 yaml_2.2.1 knitr_1.30
[86] fs_1.5.0 zip_2.1.1 nlme_3.1-150 RBGL_1.66.0 sparseMatrixStats_1.2.0
[91] xml2_1.3.2 compiler_4.0.3 rstudioapi_0.13 beeswarm_0.2.3 curl_4.3
[96] png_0.1-7 reprex_0.3.0 statmod_1.4.35 stringi_1.5.3 RSpectra_0.16-0
[101] lattice_0.20-41 nloptr_1.2.2.2 vctrs_0.3.5 pillar_1.4.6 lifecycle_0.2.0
[106] GlobalOptions_0.1.2 RcppAnnoy_0.0.17 BiocNeighbors_1.8.1 irlba_2.3.3 data.table_1.13.2
[111] bitops_1.0-6 R6_2.5.0 latticeExtra_0.6-29 gridExtra_2.3 RProtoBufLib_2.2.0
[116] rio_0.5.16 vipor_0.4.5 codetools_0.2-18 boot_1.3-25 MASS_7.3-53
[121] gtools_3.8.2 assertthat_0.2.1 rjson_0.2.20 withr_2.3.0 multcomp_1.4-15
[126] GenomeInfoDbData_1.2.4 hms_0.5.3 ncdfFlow_2.36.0 grid_4.0.3 beachmat_2.6.1
[131] minqa_1.2.4 DelayedMatrixStats_1.12.0 carData_3.0-4 Cairo_1.5-12.2 Rtsne_0.15
[136] lubridate_1.7.9.2 base64enc_0.1-3 ggbeeswarm_0.6.0**

HelenaLC commented 3 years ago

Tricky to track down the issue from this- could you produce the error again, run traceback() right after, and post the output here? Hopefully that'll help a bit, thanks!

carl-coyle commented 3 years ago

Hi there,

Thanks so much for your response. Please see below the traceback.

[cid:E1D5EBFC-5404-4531-B2F8-365E5A13B3AF@kcl.ac.uk] On 24 Nov 2020, at 12:47, Helena L. Crowell notifications@github.com<mailto:notifications@github.com> wrote:

Tricky to track down the issue from this- could you produce the error again, run traceback() right after, and post the output here? Hopefully that'll help a bit, thanks!

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FHelenaLC%2FCATALYST%2Fissues%2F164%23issuecomment-732952094&data=04%7C01%7Ccarl.coyle%40kcl.ac.uk%7C3f1cce2705234672afbc08d890770b02%7C8370cf1416f34c16b83c724071654356%7C0%7C0%7C637418188257205359%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=fjIhuDT3JF3%2F%2FqvRCzJ2ptv0KLckbwU0bsIfsPkisrM%3D&reserved=0, or unsubscribehttps://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAOST45IINLU54A7QI7IQNGTSROTMLANCNFSM4UA2EZ2A&data=04%7C01%7Ccarl.coyle%40kcl.ac.uk%7C3f1cce2705234672afbc08d890770b02%7C8370cf1416f34c16b83c724071654356%7C0%7C0%7C637418188257215355%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=6ZiGhCrsrHL8W4dxYzeeRpOq985EyGH5rDQr0Wiytew%3D&reserved=0.

HelenaLC commented 3 years ago

I think you forgot to send the traceback() output?

carl-coyle commented 3 years ago

Oops sorry. I thought I sent it as an image.

See test output below:

traceback() 6: (function(x, i, exact) if (is.matrix(i)) as.matrix(x)[[i]] else .subset2(x, i, exact = exact))(x, ..., exact = exact) 5: [[.data.frame(y, y_cols$target) 4: y[[y_cols$target]] 3: as.vector(y) 2: intersect(rownames(x), y[[y_cols$target]]) 1: plotDiffHeatmap(x = sce, y = rowData(da_res2$res))

On 24 Nov 2020, at 13:39, Helena L. Crowell notifications@github.com<mailto:notifications@github.com> wrote:

I think you forgot to send the traceback() output?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FHelenaLC%2FCATALYST%2Fissues%2F164%23issuecomment-732978457&data=04%7C01%7Ccarl.coyle%40kcl.ac.uk%7C15ea768444eb42ed5d1c08d8907e4f84%7C8370cf1416f34c16b83c724071654356%7C0%7C0%7C637418219457863410%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=CyaZRl257erXnUxR3XB%2B96uF74tG2%2B7zOEepAMfuzec%3D&reserved=0, or unsubscribehttps://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAOST45J5FFP4262A6UWDEWLSROZPJANCNFSM4UA2EZ2A&data=04%7C01%7Ccarl.coyle%40kcl.ac.uk%7C15ea768444eb42ed5d1c08d8907e4f84%7C8370cf1416f34c16b83c724071654356%7C0%7C0%7C637418219457863410%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=eZdCKnEyEwfg%2Bb1YtfDBK9JsDaU7xSGRgfpTbrULDys%3D&reserved=0.

HelenaLC commented 3 years ago

Okay, getting close- could you also please post the output of 

sce
head(rowData(da_res2$res))
carl-coyle commented 3 years ago

sce class: SingleCellExperiment dim: 41 4476877 metadata(4): experiment_info cluster_codes SOM_codes delta_area assays(2): counts exprs rownames(41): CD16 CD19 ... HLA-DR CD56 rowData names(4): channel_name marker_name marker_class used_for_clustering colnames: NULL colData names(4): sample_id condition patient_id cluster_id reducedDimNames(1): UMAP altExpNames(0):

head(rowData(da_res2$res)) DataFrame with 6 rows and 3 columns cluster_id p_val p_adj

B_Cell_CD16+_CD185+IGD+CD38+ B_Cell_CD16+_CD185+IGD+CD38+ 0.1914092 0.4029668 B_Cell_CD185+IGD+CD38+ B_Cell_CD185+IGD+CD38+ 0.1184251 0.3028183 B_Cell_CD24+CD185+_CD25Dim B_Cell_CD24+CD185+_CD25Dim 0.0196324 0.0654414 B_Cell_CD24+CD185neg_CD25Dim B_Cell_CD24+CD185neg_CD25Dim 0.2507808 0.4179680 B_Cell_CD24+IGD+CD38neg_CD185+CD25Dim B_Cell_CD24+IGD+CD38neg_CD185+CD25Dim 0.5057959 0.6976495 B_Cell_CD24+IGD+CD38neg_CD185negCD25+ B_Cell_CD24+IGD+CD38neg_CD185negCD25+ 0.8771161 0.8912320 On 25 Nov 2020, at 10:44, Helena L. Crowell > wrote:

Okay, getting close- could you also please post the output of

sce head(rowData(da_res2$res))

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FHelenaLC%2FCATALYST%2Fissues%2F164%23issuecomment-733626511&data=04%7C01%7Ccarl.coyle%40kcl.ac.uk%7C4be50788725540fb1b7108d8912f0c55%7C8370cf1416f34c16b83c724071654356%7C0%7C0%7C637418978537997901%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=KkUqyPBbjsyXbfR7gHiCMHwsC78jzCpqn1qqgMW0fmc%3D&reserved=0, or unsubscribehttps://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAOST45OLMJQWGQEQR7P2CWTSRTNXVANCNFSM4UA2EZ2A&data=04%7C01%7Ccarl.coyle%40kcl.ac.uk%7C4be50788725540fb1b7108d8912f0c55%7C8370cf1416f34c16b83c724071654356%7C0%7C0%7C637418978538007897%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=69VFJY6RC%2FMVXw7Qlf70f7XY43%2BCwNrcxnRneSddxAI%3D&reserved=0.

HelenaLC commented 3 years ago

Could you try evaluating the following and tell me what it returns (TRUE or FALSE)? It seems the function enters an if-statement that it shouldn't, resulting in the data being processed as DS results (rather than DA results).

x <- sce
y <- rowData(da_res2$res)

kids <- levels(y$cluster_id)
same <- vapply(cluster_codes(x), function(u) identical(levels(u), kids), logical(1))
k <- names(cluster_codes(x))[same][1]
x$cluster_id <- cluster_ids(x, k)

y <- data.frame(y, check.names = FALSE)
y <- dplyr::mutate_if(y, is.factor, as.character)
any(rownames(x) %in% unlist(y))
carl-coyle commented 3 years ago

The code didn’t work with the following y <- rowData(da_res2)$res so I changed it to y <- rowData(da_res2$res). I don’t know if that affects the results.

Below is the output of the code. It seems to return TRUE

x <- sce y <- rowData(da_res2$res)

kids <- levels(y$cluster_id) same <- vapply(cluster_codes(x), function(u) identical(levels(u), kids), logical(1)) k <- names(cluster_codes(x))[same][1] x$cluster_id <- cluster_ids(x, k)

y <- data.frame(y, check.names = FALSE) y <- dplyr::mutate_if(y, is.factor, as.character) any(rownames(x) %in% unlist(y)) [1] TRUE On 25 Nov 2020, at 13:22, Helena L. Crowell notifications@github.com<mailto:notifications@github.com> wrote:

Could you try evaluating the following and telling me what it returns (TRUE or FALSE)? It seams the function enters an if-statement that it shouldn't, resulting the data to be processed as DS results (rather than DA results).

x <- sce y <- rowData(da_res2)$res

kids <- levels(y$cluster_id) same <- vapply(cluster_codes(x), function(u) identical(levels(u), kids), logical(1)) k <- names(cluster_codes(x))[same][1] x$cluster_id <- cluster_ids(x, k)

y <- data.frame(y, check.names = FALSE) y <- dplyr::mutate_if(y, is.factor, as.character) any(rownames(x) %in% unlist(y))

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FHelenaLC%2FCATALYST%2Fissues%2F164%23issuecomment-733702895&data=04%7C01%7Ccarl.coyle%40kcl.ac.uk%7C431495c682c04bd5eb0108d891451f67%7C8370cf1416f34c16b83c724071654356%7C0%7C0%7C637419073347061207%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=MVLe4D8W3YO3npfhNUBoWGseD3ermRbTbpitUWBCoAM%3D&reserved=0, or unsubscribehttps://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAOST45OUXSE6FB7F2ELQ7STSRUAIFANCNFSM4UA2EZ2A&data=04%7C01%7Ccarl.coyle%40kcl.ac.uk%7C431495c682c04bd5eb0108d891451f67%7C8370cf1416f34c16b83c724071654356%7C0%7C0%7C637419073347061207%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=EW%2FuCTPQo%2Flf24wc4XtEcPWRXUpQiMVwtlL3dr8lwU4%3D&reserved=0.

HelenaLC commented 3 years ago

Yes, your fix is correct- my apologies. So the bug occurs for the following reason:

I will think about how to fix this and make the interpretation of results more accurate and robust to such occurrences. Meanwhile, I'd suggest re-labeling the cluster(s) such that they are distinguishable from the marker names. The function should then run smoothly; please let me know if so / if not!

carl-coyle commented 3 years ago

Thats makes perfect and yes that is correct one of the clusters was labelled only CD8. I have changed this now and it worked.

Thank you so much for your time on this. It is very much appreciated.

Carl.

On 25 Nov 2020, at 13:32, Helena L. Crowell notifications@github.com<mailto:notifications@github.com> wrote:

Yes, your fix is correct- my apologies. So the bug occurs for the following reason:

I will think about how to fix this and make the interpretation of results more accurate and robust to such occurrences. Meanwhile, I'd suggest re-labeling the cluster(s) such that they are distinguishable from the marker names. The function should then run smoothly.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FHelenaLC%2FCATALYST%2Fissues%2F164%23issuecomment-733708257&data=04%7C01%7Ccarl.coyle%40kcl.ac.uk%7C0ad3f0c8e3704e1219a308d89146880d%7C8370cf1416f34c16b83c724071654356%7C0%7C0%7C637419079397775630%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=G3CsVqodW8kCh44kM0WhEptxIWow6hYbYps%2FBHiwrSU%3D&reserved=0, or unsubscribehttps://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAOST45O3Z2PGRYLR5IDD6HLSRUBODANCNFSM4UA2EZ2A&data=04%7C01%7Ccarl.coyle%40kcl.ac.uk%7C0ad3f0c8e3704e1219a308d89146880d%7C8370cf1416f34c16b83c724071654356%7C0%7C0%7C637419079397785620%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=nRacj%2B0wFrUglKC9WWjaKkh0YQuLUUYQt1yoGU0tE9U%3D&reserved=0.