tomashhurst
commented
3 years ago Hi Ziyuan,
Thanks for reaching out! Do you know which version of Spectre you are using? The latest version should work OK with the updated FlowSOM/CytoNorm.
Tom
Closed ziyuanhe closed 2 years ago
tomashhurst
commented
3 years ago Hi Ziyuan,
Thanks for reaching out! Do you know which version of Spectre you are using? The latest version should work OK with the updated FlowSOM/CytoNorm.
Tom
ziyuanhe
commented
3 years ago Here is my session info: R version 4.1.0 (2021-05-18) Platform: x86_64-conda-linux-gnu (64-bit) Running under: Ubuntu 20.04.2 LTS
Matrix products: default BLAS/LAPACK: /home/jupyter/libs/r_flow/lib/libopenblasp-r0.3.15.so
locale:
[1] LC_CTYPE=C.UTF-8 LC_NUMERIC=C LC_TIME=C.UTF-8
[4] LC_COLLATE=C.UTF-8 LC_MONETARY=C.UTF-8 LC_MESSAGES=C.UTF-8
[7] LC_PAPER=C.UTF-8 LC_NAME=C LC_ADDRESS=C
[10] LC_TELEPHONE=C LC_MEASUREMENT=C.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] parallel stats graphics grDevices utils datasets methods
[8] base
other attached packages:
[1] CytoNorm_0.0.6 dtplyr_1.1.0 forcats_0.5.1
[4] stringr_1.4.0 purrr_0.3.4 readr_1.4.0
[7] tibble_3.1.2 tidyverse_1.3.1 FlowSOM_2.0.0
[10] igraph_1.2.6 flowViz_1.56.0 Biobase_2.52.0
[13] BiocGenerics_0.38.0 flowCore_2.4.0 class_7.3-19
[16] caret_6.0-88 lattice_0.20-44 ggpubr_0.4.0
[19] pheatmap_1.0.12 ggpointdensity_0.1.0 gridExtra_2.3
[22] RColorBrewer_1.1-2 colorRamps_2.3 scales_1.1.1
[25] ggthemes_4.2.4 ggplot2_3.3.5 reticulate_1.20
[28] umap_0.2.7.0 Rtsne_0.15 rstudioapi_0.13
[31] tidyr_1.1.3 dplyr_1.0.7 plyr_1.8.6
[34] data.table_1.14.0 devtools_2.4.2 usethis_2.0.1
[37] Spectre_0.5.5
loaded via a namespace (and not attached):
[1] readxl_1.3.1 uuid_0.1-4
[3] backports_1.2.1 ConsensusClusterPlus_1.56.0
[5] repr_1.1.3 splines_4.1.0
[7] scattermore_0.7 digest_0.6.27
[9] foreach_1.5.1 htmltools_0.5.1.1
[11] fansi_0.5.0 magrittr_2.0.1
[13] memoise_2.0.0 CytoML_2.4.0
[15] cluster_2.1.2 aws.signature_0.6.0
[17] openxlsx_4.2.4 remotes_2.4.0
[19] recipes_0.1.16 modelr_0.1.8
[21] gower_0.2.2 RcppParallel_5.1.4
[23] matrixStats_0.59.0 flowWorkspace_4.4.0
[25] askpass_1.1 cytolib_2.4.0
[27] prettyunits_1.1.1 jpeg_0.1-8.1
[29] colorspace_2.0-2 rvest_1.0.0
[31] ggrepel_0.9.1 haven_2.4.1
[33] callr_3.7.0 crayon_1.4.1
[35] jsonlite_1.7.2 hexbin_1.28.2
[37] graph_1.70.0 survival_3.2-11
[39] iterators_1.0.13 glue_1.4.2
[41] polyclip_1.10-0 gtable_0.3.0
[43] zlibbioc_1.38.0 ipred_0.9-11
[45] ggcyto_1.20.0 car_3.0-11
[47] pkgbuild_1.2.0 IDPmisc_1.1.20
[49] Rgraphviz_2.36.0 abind_1.4-5
[51] DBI_1.1.1 rstatix_0.7.0
[53] Rcpp_1.0.7 foreign_0.8-81
[55] stats4_4.1.0 lava_1.6.9
[57] prodlim_2019.11.13 httr_1.4.2
[59] ellipsis_0.3.2 farver_2.1.0
[61] XML_3.99-0.6 pkgconfig_2.0.3
[63] dbplyr_2.1.1 nnet_7.3-16
[65] utf8_1.2.1 tidyselect_1.1.1
[67] rlang_0.4.11 reshape2_1.4.4
[69] munsell_0.5.0 cellranger_1.1.0
[71] tools_4.1.0 cachem_1.0.5
[73] cli_3.0.0 generics_0.1.0
[75] broom_0.7.8 aws.s3_0.3.21
[77] evaluate_0.14 fastmap_1.1.0
[79] yaml_2.2.1 ModelMetrics_1.2.2.2
[81] processx_3.5.2 fs_1.5.0
[83] zip_2.2.0 RBGL_1.68.0
[85] nlme_3.1-152 xml2_1.3.2
[87] compiler_4.1.0 curl_4.3.2
[89] png_0.1-7 testthat_3.0.4
[91] ggsignif_0.6.2 reprex_2.0.0
[93] tweenr_1.0.2 stringi_1.6.2
[95] ps_1.6.0 desc_1.3.0
[97] RSpectra_0.16-0 IRdisplay_1.0
[99] Matrix_1.3-4 vctrs_0.3.8
[101] pillar_1.6.1 lifecycle_1.0.0
[103] R6_2.5.0 latticeExtra_0.6-29
[105] KernSmooth_2.23-20 RProtoBufLib_2.4.0
[107] rio_0.5.27 sessioninfo_1.1.1
[109] codetools_0.2-18 MASS_7.3-54
[111] assertthat_0.2.1 pkgload_1.2.1
[113] openssl_1.4.4 rprojroot_2.0.2
[115] withr_2.4.2 S4Vectors_0.30.0
[117] ncdfFlow_2.38.0 hms_1.1.0
[119] grid_4.1.0 rpart_4.1-15
[121] timeDate_3043.102 IRkernel_1.2
[123] carData_3.0-4 ggnewscale_0.4.5
[125] ggforce_0.3.3 pbdZMQ_0.3-5
[127] pROC_1.17.0.1 lubridate_1.7.10
[129] base64enc_0.1-3
ziyuanhe
commented
3 years ago Hi Tom,
I updated all the related packages again and still getting this error Error in flowCore::exprs(ff)[, channels, drop = FALSE]: subscript out of bounds when running this function train.cytonorm .
From what i can tell, error comes from this line clusterRes[[cluster]] <- do.call(normMethod.train, normParams_tmp) when cellular.cols and cluster.cols are different when generating prep.cytonorm object. I can see the output temporary fcs file per flowsom cluster (e.g. 1_fsom1.fcs) only contains the features in cluster.cols which were used for flowsom but not other features in cellular.cols. So it caused normMethod.train function not able to those columns. Do you have any insights how to fix this?
Thanks, Ziyuan
tomashhurst
commented
3 years ago Hi @ziyuanhe,
That's some excellent detective work! I believe that all of the cluster.cols need to be included in cellular.cols, as I think there is some subsetting that depends on this assumption. Is that the case in your data? In theory the two should be allowed to be different, but one has to be included within the other.
hrj21
commented
3 years ago Hiya,
I seem to be having the same issue with Spectre 0.5.5. When I redefine prep.cytonorm() to include fsom$FlowSOM$data and fsom$FlowSOM$map$mapping the function runs fine. I have removed and then reinstalled Spectre from github, but I get the same behaviour.
EDIT: I have similarly had to redefine the train.cytonorm() and run.cytonorm() functions to use fsom$FlowSOM$, in order for them to work.
sessioninfo::package_info()
! package * version date lib source
abind 1.4-5 2016-07-21 [1] CRAN (R 4.0.3)
askpass 1.1 2019-01-13 [1] CRAN (R 4.0.3)
assertthat 0.2.1 2019-03-21 [1] CRAN (R 4.0.3)
aws.s3 0.3.21 2020-04-07 [1] CRAN (R 4.0.3)
aws.signature 0.6.0 2020-06-01 [1] CRAN (R 4.0.3)
backports 1.2.1 2020-12-09 [1] CRAN (R 4.0.3)
base64enc 0.1-3 2015-07-28 [1] CRAN (R 4.0.3)
Biobase * 2.50.0 2020-10-27 [1] Bioconductor
BiocGenerics * 0.36.1 2021-04-16 [1] Bioconductor
BiocManager 1.30.16 2021-06-15 [1] CRAN (R 4.0.5)
broom 0.7.9 2021-07-27 [1] CRAN (R 4.0.2)
cachem 1.0.5 2021-05-15 [1] CRAN (R 4.0.5)
callr 3.7.0 2021-04-20 [1] CRAN (R 4.0.5)
car 3.0-11 2021-06-27 [1] CRAN (R 4.0.5)
carData 3.0-4 2020-05-22 [1] CRAN (R 4.0.3)
caret * 6.0-88 2021-05-15 [1] CRAN (R 4.0.5)
cellranger 1.1.0 2016-07-27 [1] CRAN (R 4.0.3)
class * 7.3-19 2021-05-03 [1] CRAN (R 4.0.5)
cli 3.0.1 2021-07-17 [1] CRAN (R 4.0.5)
cluster 2.1.2 2021-04-17 [1] CRAN (R 4.0.5)
codetools 0.2-18 2020-11-04 [1] CRAN (R 4.0.3)
colorRamps * 2.3 2012-10-29 [1] CRAN (R 4.0.3)
colorspace 2.0-2 2021-06-24 [1] CRAN (R 4.0.5)
ConsensusClusterPlus 1.54.0 2020-10-27 [1] Bioconductor
crayon 1.4.1 2021-02-08 [1] CRAN (R 4.0.5)
curl 4.3.2 2021-06-23 [1] CRAN (R 4.0.5)
cytolib 2.2.1 2021-01-17 [1] Bioconductor
CytoML 2.2.2 2021-03-10 [1] Bioconductor
CytoNorm * 0.0.5 2021-01-07 [1] Github (saeyslab/CytoNorm@e94732c)
data.table * 1.14.0 2021-02-21 [1] CRAN (R 4.0.5)
DBI 1.1.1 2021-01-15 [1] CRAN (R 4.0.5)
desc 1.3.0 2021-03-05 [1] CRAN (R 4.0.5)
devtools * 2.4.2 2021-06-07 [1] CRAN (R 4.0.5)
digest 0.6.27 2020-10-24 [1] CRAN (R 4.0.3)
dplyr * 1.0.7 2021-06-18 [1] CRAN (R 4.0.5)
ellipsis 0.3.2 2021-04-29 [1] CRAN (R 4.0.5)
factoextra * 1.0.7 2020-04-01 [1] CRAN (R 4.0.5)
fansi 0.5.0 2021-05-25 [1] CRAN (R 4.0.5)
fastmap 1.1.0 2021-01-25 [1] CRAN (R 4.0.5)
flowCore * 2.2.0 2020-10-27 [1] Bioconductor
FlowSOM * 1.22.0 2020-10-27 [1] Bioconductor
flowViz * 1.54.0 2020-10-27 [1] Bioconductor
flowWorkspace 4.2.0 2020-10-27 [1] Bioconductor
forcats 0.5.1 2021-01-27 [1] CRAN (R 4.0.5)
foreach 1.5.1 2020-10-15 [1] CRAN (R 4.0.3)
foreign 0.8-81 2020-12-22 [1] CRAN (R 4.0.3)
fs 1.5.0 2020-07-31 [1] CRAN (R 4.0.5)
generics 0.1.0 2020-10-31 [1] CRAN (R 4.0.3)
ggcyto 1.18.0 2020-10-27 [1] Bioconductor
ggplot2 * 3.3.5 2021-06-25 [1] CRAN (R 4.0.5)
ggpointdensity * 0.1.0 2019-08-28 [1] CRAN (R 4.0.3)
ggpubr * 0.4.0 2020-06-27 [1] CRAN (R 4.0.3)
ggrepel 0.9.1 2021-01-15 [1] CRAN (R 4.0.5)
ggsignif 0.6.2 2021-06-14 [1] CRAN (R 4.0.5)
ggthemes * 4.2.4 2021-01-20 [1] CRAN (R 4.0.5)
glue 1.4.2 2020-08-27 [1] CRAN (R 4.0.3)
gower 0.2.2 2020-06-23 [1] CRAN (R 4.0.3)
graph 1.68.0 2020-10-27 [1] Bioconductor
gridExtra * 2.3 2017-09-09 [1] CRAN (R 4.0.3)
gtable 0.3.0 2019-03-25 [1] CRAN (R 4.0.3)
haven 2.4.3 2021-08-04 [1] CRAN (R 4.0.5)
hexbin 1.28.2 2021-01-08 [1] CRAN (R 4.0.5)
hms 1.1.0 2021-05-17 [1] CRAN (R 4.0.5)
httr 1.4.2 2020-07-20 [1] CRAN (R 4.0.3)
IDPmisc 1.1.20 2020-01-21 [1] CRAN (R 4.0.3)
igraph * 1.2.6 2020-10-06 [1] CRAN (R 4.0.3)
ipred 0.9-11 2021-03-12 [1] CRAN (R 4.0.5)
iterators 1.0.13 2020-10-15 [1] CRAN (R 4.0.3)
jpeg 0.1-9 2021-07-24 [1] CRAN (R 4.0.5)
jsonlite 1.7.2 2020-12-09 [1] CRAN (R 4.0.5)
KernSmooth 2.23-20 2021-05-03 [1] CRAN (R 4.0.5)
lattice * 0.20-41 2020-04-02 [2] CRAN (R 4.0.2)
latticeExtra 0.6-29 2019-12-19 [1] CRAN (R 4.0.3)
lava 1.6.9 2021-03-11 [1] CRAN (R 4.0.5)
lifecycle 1.0.0 2021-02-15 [1] CRAN (R 4.0.5)
lubridate 1.7.10 2021-02-26 [1] CRAN (R 4.0.5)
magrittr 2.0.1 2020-11-17 [1] CRAN (R 4.0.3)
MASS 7.3-54 2021-05-03 [1] CRAN (R 4.0.5)
Matrix 1.3-4 2021-06-01 [1] CRAN (R 4.0.5)
matrixStats 0.60.0 2021-07-26 [1] CRAN (R 4.0.2)
memoise 2.0.0 2021-01-26 [1] CRAN (R 4.0.5)
ModelMetrics 1.2.2.2 2020-03-17 [1] CRAN (R 4.0.3)
munsell 0.5.0 2018-06-12 [1] CRAN (R 4.0.3)
ncdfFlow 2.36.0 2020-10-27 [1] Bioconductor
nlme 3.1-152 2021-02-04 [1] CRAN (R 4.0.5)
nnet 7.3-16 2021-05-03 [1] CRAN (R 4.0.5)
openssl 1.4.4 2021-04-30 [1] CRAN (R 4.0.5)
openxlsx 4.2.4 2021-06-16 [1] CRAN (R 4.0.5)
pheatmap * 1.0.12 2019-01-04 [1] CRAN (R 4.0.3)
pillar 1.6.2 2021-07-29 [1] CRAN (R 4.0.2)
pkgbuild 1.2.0 2020-12-15 [1] CRAN (R 4.0.5)
pkgconfig 2.0.3 2019-09-22 [1] CRAN (R 4.0.3)
pkgload 1.2.1 2021-04-06 [1] CRAN (R 4.0.2)
plyr * 1.8.6 2020-03-03 [1] CRAN (R 4.0.3)
png 0.1-7 2013-12-03 [1] CRAN (R 4.0.3)
prettyunits 1.1.1 2020-01-24 [1] CRAN (R 4.0.3)
pROC 1.17.0.1 2021-01-13 [1] CRAN (R 4.0.3)
processx 3.5.2 2021-04-30 [1] CRAN (R 4.0.5)
prodlim 2019.11.13 2019-11-17 [1] CRAN (R 4.0.3)
ps 1.6.0 2021-02-28 [1] CRAN (R 4.0.5)
purrr 0.3.4 2020-04-17 [1] CRAN (R 4.0.3)
R6 2.5.0 2020-10-28 [1] CRAN (R 4.0.3)
RBGL 1.66.0 2020-10-27 [1] Bioconductor
RColorBrewer * 1.1-2 2014-12-07 [1] CRAN (R 4.0.3)
Rcpp 1.0.7 2021-07-07 [1] CRAN (R 4.0.5)
D RcppParallel 5.1.4 2021-05-04 [1] CRAN (R 4.0.5)
readxl 1.3.1 2019-03-13 [1] CRAN (R 4.0.3)
recipes 0.1.16 2021-04-16 [1] CRAN (R 4.0.5)
remotes 2.4.0 2021-06-02 [1] CRAN (R 4.0.5)
reshape2 1.4.4 2020-04-09 [1] CRAN (R 4.0.3)
reticulate * 1.20 2021-05-03 [1] CRAN (R 4.0.5)
Rgraphviz 2.34.0 2020-10-27 [1] Bioconductor
rio 0.5.27 2021-06-21 [1] CRAN (R 4.0.5)
rlang 0.4.11 2021-04-30 [1] CRAN (R 4.0.5)
rpart 4.1-15 2019-04-12 [2] CRAN (R 4.0.2)
rprojroot 2.0.2 2020-11-15 [1] CRAN (R 4.0.3)
RProtoBufLib 2.2.0 2020-10-27 [1] Bioconductor
RSpectra 0.16-0 2019-12-01 [1] CRAN (R 4.0.3)
rstatix 0.7.0 2021-02-13 [1] CRAN (R 4.0.5)
rstudioapi * 0.13 2020-11-12 [1] CRAN (R 4.0.3)
Rtsne * 0.15 2018-11-10 [1] CRAN (R 4.0.3)
S4Vectors 0.28.1 2020-12-09 [1] Bioconductor
scales * 1.1.1 2020-05-11 [1] CRAN (R 4.0.3)
sessioninfo 1.1.1 2018-11-05 [1] CRAN (R 4.0.3)
Spectre * 0.5.5 2021-08-18 [1] Github (immunedynamics/spectre@742ebc4)
stringi 1.7.3 2021-07-16 [1] CRAN (R 4.0.2)
stringr 1.4.0 2019-02-10 [1] CRAN (R 4.0.3)
survival 3.2-11 2021-04-26 [1] CRAN (R 4.0.5)
testthat 3.0.4 2021-07-01 [1] CRAN (R 4.0.5)
tibble 3.1.3 2021-07-23 [1] CRAN (R 4.0.2)
tidyr * 1.1.3 2021-03-03 [1] CRAN (R 4.0.5)
tidyselect 1.1.1 2021-04-30 [1] CRAN (R 4.0.5)
timeDate 3043.102 2018-02-21 [1] CRAN (R 4.0.3)
tsne 0.1-3 2016-07-15 [1] CRAN (R 4.0.3)
umap * 0.2.7.0 2020-11-04 [1] CRAN (R 4.0.3)
usethis * 2.0.1 2021-02-10 [1] CRAN (R 4.0.5)
utf8 1.2.2 2021-07-24 [1] CRAN (R 4.0.2)
vctrs 0.3.8 2021-04-29 [1] CRAN (R 4.0.5)
withr 2.4.2 2021-04-18 [1] CRAN (R 4.0.5)
XML 3.99-0.6 2021-03-16 [1] CRAN (R 4.0.5)
xml2 1.3.2 2020-04-23 [1] CRAN (R 4.0.3)
yaml 2.2.1 2020-02-01 [1] CRAN (R 4.0.3)
zip 2.2.0 2021-05-31 [1] CRAN (R 4.0.5)
zlibbioc 1.36.0 2020-10-28 [1] Bioconductor
[1] C:/Users/u061745/OneDrive - UCB/Documents/R/win-library/4.0
[2] C:/Program Files/R/R-4.0.2/library@hrj21 @ziyuanhe Ah, I think you need to update FlowSOM to v2.0.0 from Bioconductor. There was a structural change to how FlowSOM stores data within the FlowSOM object, which is present in v2.0.0 and above.
You can use the following to re-install it at the current version (v2.0.0):
if (!requireNamespace("BiocManager", quietly = TRUE))
install.packages("BiocManager")
BiocManager::install("FlowSOM")Or you can bulk update your installed Bioconductor packages.
hrj21
commented
3 years ago That seems to have solved it! Sorry for that. Weirdly, installing with BiocManager::install("FlowSOM") kept installing version 1.2.2. In the end I downloaded the windows binary and asked RStudio to install it from the .zip file.
tomashhurst
commented
3 years ago Huh, that's weird.
tomashhurst
commented
3 years ago Oh you might need to update your 'bioconductor' version, as the release of the individual package versions are tied to the version of bioconductor too. Some info on here: https://cran.r-project.org/web/packages/BiocManager/vignettes/BiocManager.html
tomashhurst
commented
3 years ago @ziyuanhe did you manage to get this sorted?
tomashhurst
commented
2 years ago @ziyuanhe I'm going to close this one for now, but please do reach out if you have any other issues!
Hi,
Thank for developing this package. it is very helpful to have the data in data.table at all time instead of all these different formats in the flow cytometry data analysis world.
I run into this issue recently to use Cytonorm with your packages. i think Cytonorm recently update their package to be compatible with the newer version of the FlowSOM. As a result, i think your prep.cytonorm and train.cytonorm function needs to update. The issue i found comes from fsom$data is now under fsom$FlowSOM$data. so wherever the function call fsom$data it return NULL. I think it should be an easy fix.
Thanks, Ziyuan