Closed bradleyed closed 3 years ago
just wanted to let you know it is on my todo list next week and will get back to you on this
Should be working now. You will need to pull the latest CytoML from github master and pull the new docker image
docker pull public.ecr.aws/x4k5d9i7/cytoverse/gs-to-wsp:latest
Great! Thanks so much for the timely fix.
It might be a bit before I can confirm that it works as expected for me-- I have previously tried to install the development versions of the cytoverse packages, but I have gotten stuck with errors on installation. From what I can surmise from other "issue" posts ( like #108 ), I think it has something to do with needing to update system libraries, which is not something I've had a lot of experience with (though I did try). I think it failed when trying to compile the "cytolib" package, and it produced more output text than the R console was willing to save.
Anyway, I will give it a try again-- perhaps I will figure out where I was going wrong before.
I was thinking, however, that it might be helpful for people with less knowledge in this area (like me) to have a concise list of all the prerequisite steps that are needed in order to successfully install the development version of the cytoverse package on a windows 10 system (assuming proficiency with R, but not with Rtools, package compilation, etc.). Something like that might help me troubleshoot where I am going wrong.
Thanks again. I have really appreciated using these packages. I will get back to you soon.
Brad
If you are on windows, you should probably install directly from bioc devel. I will try to fix windows build error on bioc server and keep you posted
CytoML_error_instal_output.txt
Hi Mike, I am happy to wait for the bioc devel when that is ready, but I thought I'd send you the output I get when I try to run _devtools::installgithub("RGLab/CytoML") --just in case you see something in there that points to a quick fix on my end.
When I run _devtools::installgithub("RGLab/cytolib") or _cytoverse::cytoverseupdate(repo = "github") the output is too long to be saved in the console, but I have attached latter portion below. Here are the packages that would be updated with _cytoverseupdate
Cytoverse_update_github_output.txt
Does this still need to be done? https://github.com/RGLab/RProtoBufLib/blob/master/INSTALL If so, then that could be my problem, as I do not completely understand it.
No. As long as Rtools4 is set up, you should be able to just do
remotes::install_github("RGLab/RProtobufLib")
Downloading GitHub repo RGLab/RProtobufLib@HEAD
√ checking for file 'C:\Users\Administrator\AppData\Local\Temp\2\Rtmpy4lgeG\remotesb247ebf147f\RGLab-RProtoBufLib-70ce6e6/DESCRIPTION' (336ms)
- preparing 'RProtoBufLib': (474ms)
√ checking DESCRIPTION meta-information ...
- cleaning src
- running 'cleanup.win'
- checking for LF line-endings in source and make files and shell scripts
- checking for empty or unneeded directories
- building 'RProtoBufLib_2.3.4.tar.gz'
Warning: file 'RProtoBufLib/cleanup' did not have execute permissions: corrected
Warning: file 'RProtoBufLib/configure' did not have execute permissions: corrected
* installing *source* package 'RProtoBufLib' ...
** using staged installation
**********************************************
WARNING: this package has a configure script
It probably needs manual configuration
**********************************************
** libs
*** arch - i386
"C:/PROGRA~1/R/R-devel/bin/i386/Rscript.exe" "../tools/downloadlibs.R" 3.13.0
"C:/rtools40/mingw32/bin/"g++ -std=gnu++11 -I"C:/PROGRA~1/R/R-devel/include" -DNDEBUG -I../inst/include/ -I../windows/protobuf-3.13.0/include -I"C:/extsoft/include" -O2 -Wall -mfpmath=sse -msse2 -mstackrealign -c GatingSet_pb_lib/GatingSet.pb.cc -o GatingSet_pb_lib/GatingSet.pb.o
mkdir -p "C:/Program Files/R/R-devel/library/00LOCK-RProtoBufLib/00new/RProtoBufLib/include"
cp -r ../windows/protobuf-3.13.0/include/google "C:/Program Files/R/R-devel/library/00LOCK-RProtoBufLib/00new/RProtoBufLib/include"
mkdir -p "C:/Program Files/R/R-devel/library/00LOCK-RProtoBufLib/00new/RProtoBufLib/lib/i386"
cp ../windows/protobuf-3.13.0/lib/i386/libprotobuf.a "C:/Program Files/R/R-devel/library/00LOCK-RProtoBufLib/00new/RProtoBufLib/lib/i386/libprotobuf.a"
cp GatingSet_pb_lib/GatingSet.pb.o "C:/Program Files/R/R-devel/library/00LOCK-RProtoBufLib/00new/RProtoBufLib/lib/i386/GatingSet.pb.o"
"C:/rtools40/mingw32/bin/"g++ -std=gnu++11 -I"C:/PROGRA~1/R/R-devel/include" -DNDEBUG -I../inst/include/ -I../windows/protobuf-3.13.0/include -I"C:/extsoft/include" -O2 -Wall -mfpmath=sse -msse2 -mstackrealign -c dummy.cpp -o dummy.o
C:/rtools40/mingw32/bin/g++ -shared -s -static-libgcc -o RProtoBufLib.dll tmp.def dummy.o -LC:/extsoft/lib/i386 -LC:/extsoft/lib -LC:/PROGRA~1/R/R-devel/bin/i386 -lR
installing to C:/Program Files/R/R-devel/library/00LOCK-RProtoBufLib/00new/RProtoBufLib/libs/i386
*** arch - x64
"C:/PROGRA~1/R/R-devel/bin/x64/Rscript.exe" "../tools/downloadlibs.R" 3.13.0
"C:/rtools40/mingw64/bin/"g++ -std=gnu++11 -I"C:/PROGRA~1/R/R-devel/include" -DNDEBUG -I../inst/include/ -I../windows/protobuf-3.13.0/include -I"C:/extsoft/include" -O2 -Wall -mfpmath=sse -msse2 -mstackrealign -c GatingSet_pb_lib/GatingSet.pb.cc -o GatingSet_pb_lib/GatingSet.pb.o
mkdir -p "C:/Program Files/R/R-devel/library/00LOCK-RProtoBufLib/00new/RProtoBufLib/include"
cp -r ../windows/protobuf-3.13.0/include/google "C:/Program Files/R/R-devel/library/00LOCK-RProtoBufLib/00new/RProtoBufLib/include"
mkdir -p "C:/Program Files/R/R-devel/library/00LOCK-RProtoBufLib/00new/RProtoBufLib/lib/x64"
cp ../windows/protobuf-3.13.0/lib/x64/libprotobuf.a "C:/Program Files/R/R-devel/library/00LOCK-RProtoBufLib/00new/RProtoBufLib/lib/x64/libprotobuf.a"
cp GatingSet_pb_lib/GatingSet.pb.o "C:/Program Files/R/R-devel/library/00LOCK-RProtoBufLib/00new/RProtoBufLib/lib/x64/GatingSet.pb.o"
"C:/rtools40/mingw64/bin/"g++ -std=gnu++11 -I"C:/PROGRA~1/R/R-devel/include" -DNDEBUG -I../inst/include/ -I../windows/protobuf-3.13.0/include -I"C:/extsoft/include" -O2 -Wall -mfpmath=sse -msse2 -mstackrealign -c dummy.cpp -o dummy.o
C:/rtools40/mingw64/bin/g++ -shared -s -static-libgcc -o RProtoBufLib.dll tmp.def dummy.o -LC:/extsoft/lib/x64 -LC:/extsoft/lib -LC:/PROGRA~1/R/R-devel/bin/x64 -lR
installing to C:/Program Files/R/R-devel/library/00LOCK-RProtoBufLib/00new/RProtoBufLib/libs/x64
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
converting help for package 'RProtoBufLib'
finding HTML links ... done
LdFlags html
REDIRECT:topic LdFlags -> LdFlags.html [ SUCCESS ]
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
*** arch - i386
*** arch - x64
** testing if installed package can be loaded from final location
*** arch - i386
*** arch - x64
** testing if installed package keeps a record of temporary installation path
* DONE (RProtoBufLib)
> packageVersion("RProtobufLib")
[1] ‘2.3.4’
Thanks Mike, yep, RProtoBufLib installed fine, just as you showed, but still failing install on development versions of cytolib and CytoML. Managed to get the beginning of the output of attempted to cytolib install here...
I don't see the error.
That's the first part of the output, the latter portion is attached at the bottom of yesterday's post, just after the screen shot. The output is so large that R doesn't save the entirety of it, so I can only grab a portion of it at a time. Sorry for the confusion.
I've pushed the patch. Can you try again?
Hi Mike, I apologize, but I am stuck at the installation of the cytolib package from source. I seem to having the same/similar problem as this person here , judging by the similarity of our output. Since you have likely fixed the original issue with gatingset_to_flowjo, should we pause this issue post until I can get the cytolib installation sorted out and I am able to try out your fix?
Ok! Finally figured out what I was doing wrong with the installation. The patch works! FJ 9 carats ("<>") are removed from the channel names, and the compensation works as expected. Thanks so much!
or else you just fixed something else that allowed it to work. Either way, thanks again.
Describe the bug I am using cytoverse packages to help with analysis of old flow data, which has been partially gated in Flowjo 9. Since FJ-9 is becoming obsolete, I am using the cytoverse packages to move the analysis to FJ-10. While I am doing this migration I decided to use the flowAI package to produce cleaned versions of all the FCS files. So a simplified workflow is, generally:
The data look great in FJ-10, reproducing exactly as things were in FJ-9. However, there is sometimes a need to tweak our compensation matrices to get the compensation just right. When I go into the compensation window in FJ-10, FJ doesn't recognize the parameters in the compensation matrix as being in the samples:
I would like to be able to tweak the matrix occasionally in FJ-10, but I can't really do that here.
To Reproduce
Created on 2021-03-03 by the reprex package (v1.0.0)
Session info
``` r sessionInfo() #> R version 4.0.3 (2020-10-10) #> Platform: x86_64-w64-mingw32/x64 (64-bit) #> Running under: Windows 10 x64 (build 19042) #> #> Matrix products: default #> #> locale: #> [1] LC_COLLATE=English_United States.1252 #> [2] LC_CTYPE=English_United States.1252 #> [3] LC_MONETARY=English_United States.1252 #> [4] LC_NUMERIC=C #> [5] LC_TIME=English_United States.1252 #> #> attached base packages: #> [1] stats graphics grDevices utils datasets methods base #> #> other attached packages: #> [1] ggcyto_1.18.0 ncdfFlow_2.36.0 BH_1.75.0-0 #> [4] RcppArmadillo_0.10.2.1.0 flowCore_2.2.0 CytoML_2.2.1 #> [7] flowWorkspace_4.2.0 forcats_0.5.1 stringr_1.4.0 #> [10] dplyr_1.0.4 purrr_0.3.4 readr_1.4.0 #> [13] tidyr_1.1.2 tibble_3.0.6 ggplot2_3.3.3 #> [16] tidyverse_1.3.0 #> #> loaded via a namespace (and not attached): #> [1] matrixStats_0.58.0 fs_1.5.0 lubridate_1.7.9.2 #> [4] RColorBrewer_1.1-2 httr_1.4.2 Rgraphviz_2.34.0 #> [7] tools_4.0.3 backports_1.2.1 R6_2.5.0 #> [10] DBI_1.1.1 BiocGenerics_0.36.0 colorspace_2.0-0 #> [13] withr_2.4.1 tidyselect_1.1.0 gridExtra_2.3 #> [16] curl_4.3 compiler_4.0.3 graph_1.68.0 #> [19] cli_2.3.0 rvest_0.3.6 Biobase_2.50.0 #> [22] xml2_1.3.2 labeling_0.4.2 scales_1.1.1 #> [25] hexbin_1.28.2 RBGL_1.66.0 digest_0.6.27 #> [28] rmarkdown_2.6 base64enc_0.1-3 jpeg_0.1-8.1 #> [31] pkgconfig_2.0.3 htmltools_0.5.1.1 styler_1.3.2 #> [34] dbplyr_2.1.0 highr_0.8 rlang_0.4.10 #> [37] readxl_1.3.1 farver_2.0.3 generics_0.1.0 #> [40] jsonlite_1.7.2 magrittr_2.0.1 RProtoBufLib_2.2.0 #> [43] Rcpp_1.0.6 munsell_0.5.0 S4Vectors_0.28.1 #> [46] lifecycle_1.0.0 stringi_1.5.3 yaml_2.2.1 #> [49] zlibbioc_1.36.0 plyr_1.8.6 grid_4.0.3 #> [52] parallel_4.0.3 crayon_1.4.1 lattice_0.20-41 #> [55] haven_2.3.1 hms_1.0.0 knitr_1.31 #> [58] pillar_1.4.7 stats4_4.0.3 reprex_1.0.0 #> [61] XML_3.99-0.5 glue_1.4.2 evaluate_0.14 #> [64] latticeExtra_0.6-29 data.table_1.13.6 RcppParallel_5.0.2 #> [67] modelr_0.1.8 png_0.1-7 vctrs_0.3.6 #> [70] cellranger_1.1.0 gtable_0.3.0 aws.s3_0.3.21 #> [73] assertthat_0.2.1 xfun_0.21 mime_0.10 #> [76] broom_0.7.4 cytolib_2.2.1 aws.signature_0.6.0 #> [79] ellipsis_0.3.1 ```I think the problem lies in the carets that FJ-9 puts around compensated parameter names. Even though the cleaned fcs file obviously does not have carets in its parameter names, gatingset_to_flowjo seems to overide the FCS file value. Below is a screenshot of the output workspace. Note the two fluorescent parameter names shown on the right.
Expected behavior I hoped samples would be exported so that compensation matrix could be edited in flowjo and applied to samples if needed
Additional context I think this problem would occur with exporting the original sample to FJ 10 as well. I left the gh_apply_to_cs() stuff in my example because it's a critical part of my workflow and I want to ensure that a proposed solution takes this into account.
I am happy to share the files in this example privately.
Thanks so much!