Open RPdavies opened 3 years ago
yeah, send the link or data to mike@ozette.ai. Thanks!
No problem, I've sent it over just now. Please let me know if the link didn't work for some reason. Thank you!
There is currently no elegant way to decide whether xml compensation should/can be used since compensation
for this xml is actually present, but as dummy setting, thus caused the parse to be confused.
<compensation>
<compensation_coefficient>0.0</compensation_coefficient>
<compensation_coefficient>1.0</compensation_coefficient>
</compensation>
I've pushed the easy workaround as you suggested, i.e. optionally disable it
gs <- diva_to_gatingset(ws, name = 1, xml_compensation_enabled = FALSE)
Awesome, that does the job. Look forward to using CytoML!
I also have a dummy compensation issue in my xml file.
I tried setting xml_compensation_enabled to either TRUE or FALSE and both times I received the same error. Are there alternative solutions to this problem?
I am wondering if it might be easier to create a function that removes dummy compensation data from diva xml files. I tried removing the code listed above from my xml file but I received the following error.
Can you show the error and traceback() output when you set xml_compensation_enabled = F
?
I don't know much about these functions, but it seems like diva_to_gatingset is not passing xml_compensation_enabled = F
to .parseDivaWorkspace when it is called. I could be misinterpreting things however.
You may need to pull the latest version from github.
I am currently having trouble running the diva_to_gatingset function from the CytoML package
I am using CytoML version 2.4.0. available through biocManager My experiment lacks any compensation, but my XML from Diva includes placeholder values. An example is shown below:
> library(flowWorkspace)
> library(CytoML)
> setwd("/Users/user1/Desktop/DummyProject_Folder/DummyExp")
> flowData <- getwd()
> gatingFile <- list.files(flowData, pattern=".xml",full = TRUE)
> ws <- open_diva_xml(gatingFile)
> ws
> diva_get_sample_groups(ws)
> gs <- diva_to_gatingset(ws)
> sampleNames(gs)
> gs_get_pop_paths(gs)
> plotGate(gs[[1]])
> sessionInfo()
Diva Workspace Version Version 9.1.2
File location: /Users/user1/Desktop/DummyProject_Folder/DummyExp
File name: DummyExp.xml
Workspace is open.
Groups in Workspace
> diva_get_sample_groups(ws)
> gs <- diva_to_gatingset(ws)
Choose which group of samples to import:
1: DummySpecimen1
> Selection: 1
Parsing 2 samples
loading data: /Users/nvannest/Desktop/ACE openCyto Test/DummyProject_Folder/DummyExp_Exp_Export/DummyExp_DummySpecimen1_DummyTube1.fcs
swap cols: FSC-H:FSC-W
swap cols: SSC-H:SSC-W
Compensating
Error in solve.default(comp) : 'a' (7 x 14) must be square )
When I try to install the version of CytoML available through github I get an installation error
> devtools::install_github("RGLab/CytoML")
Downloading GitHub repo RGLab/CytoML@HEAD
These packages have more recent versions available.
It is recommended to update all of them.
Which would you like to update?
1: All
2: CRAN packages only
3: None
4: robustbase (0.93-7 -> 0.93-8) [CRAN]
5: matrixStats (0.58.0 -> 0.59.0) [CRAN]
6: ks (1.13.0 -> 1.13.1) [CRAN]
> Enter one or more numbers, or an empty line to skip updates: 1
robustbase (0.93-7 -> 0.93-8) [CRAN]
matrixStats (0.58.0 -> 0.59.0) [CRAN]
ks (1.13.0 -> 1.13.1) [CRAN]
Installing 3 packages: robustbase, matrixStats, ks
There are binary versions available but the source versions are later:
binary source needs_compilation
robustbase 0.93-7 0.93-8 TRUE
matrixStats 0.58.0 0.59.0 TRUE
ks 1.13.0 1.13.1 TRUE
> Do you want to install from sources the packages which need compilation? (Yes/no/cancel) no
trying URL 'https://cran.rstudio.com/bin/macosx/contrib/4.1/robustbase_0.93-7.tgz'
Content type 'application/x-gzip' length 3123297 bytes (3.0 MB)
==================================================
downloaded 3.0 MB
trying URL 'https://cran.rstudio.com/bin/macosx/contrib/4.1/matrixStats_0.58.0.tgz'
Content type 'application/x-gzip' length 2019790 bytes (1.9 MB)
==================================================
downloaded 1.9 MB
trying URL 'https://cran.rstudio.com/bin/macosx/contrib/4.1/ks_1.13.0.tgz'
Content type 'application/x-gzip' length 1851632 bytes (1.8 MB)
==================================================
downloaded 1.8 MB
The downloaded binary packages are in
/var/folders/_g/xp8hp7mn02z122v2092lzw499zps22/T//RtmpcPvBBy/downloaded_packages
✓ checking for file ‘/private/var/folders/_g/xp8hp7mn02z122v2092lzw499zps22/T/RtmpcPvBBy/remotes132e432e8405d/RGLab-CytoML-d544a00/DESCRIPTION’ ...
─ preparing ‘CytoML’:
✓ checking DESCRIPTION meta-information ...
─ cleaning src
─ checking for LF line-endings in source and make files and shell scripts
─ checking for empty or unneeded directories
Omitted ‘LazyData’ from DESCRIPTION
NB: this package now depends on R (>= 3.5.0)
WARNING: Added dependency on R >= 3.5.0 because serialized objects in
serialize/load version 3 cannot be read in older versions of R.
File(s) containing such objects:
‘CytoML/tests/testthat/expect_result/flowJoWorkspace_expect.rds’
─ building ‘CytoML_2.5.1.tar.gz’
Warning in sprintf(gettext(fmt, domain = domain), ...) :
one argument not used by format 'invalid uid value replaced by that for user 'nobody''
Warning: invalid uid value replaced by that for user 'nobody'
Warning in sprintf(gettext(fmt, domain = domain), ...) :
one argument not used by format 'invalid gid value replaced by that for user 'nobody''
Warning: invalid gid value replaced by that for user 'nobody'
* installing *source* package ‘CytoML’ ...
** using staged installation
checking whether the C++ compiler works... yes
checking for C++ compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C++ compiler... yes
checking whether clang++ -mmacosx-version-min=10.13 -std=gnu++11 accepts -g... yes
checking for gcc... clang -mmacosx-version-min=10.13
checking whether we are using the GNU C compiler... yes
checking whether clang -mmacosx-version-min=10.13 accepts -g... yes
checking for clang -mmacosx-version-min=10.13 option to accept ISO C89... none needed
configure: setting xml2 flags...
configure: No directory was specified for --with-xml2. Trying to find xml2 using other methods.
checking for xml2-config... /usr/bin/xml2-config
configure: setting cytolib-ml commandline tool path...
configure: Using the following compilation and linking flags
configure: PKG_CPPFLAGS=-I/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include -ftemplate-depth=900
configure: PKG_LIBS=-L/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/lib -lxml2 -lz -lpthread -licucore -lm
configure: CYTOLIBML_BIN=/usr/local/bin
configure: creating ./config.status
config.status: creating src/Makevars
config.status: creating R/cytolibml_bin_path.R
** libs
clang++ -mmacosx-version-min=10.13 -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -DROUT -I../inst/include/ -I/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include -ftemplate-depth=900 -DBOOST_NO_AUTO_PTR -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/BH/include' -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/RProtoBufLib/include' -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/cytolib/include' -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rhdf5lib/include' -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/RcppArmadillo/include' -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/RcppParallel/include' -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/flowWorkspace/include' -I/usr/local/include -fPIC -Wall -g -O2 -c RcppExports.cpp -o RcppExports.o
In file included from RcppExports.cpp:4:
In file included from ./../inst/include/CytoML.h:4:
In file included from /Library/Frameworks/R.framework/Versions/4.1/Resources/library/flowWorkspace/include/flowWorkspace.h:5:
In file included from /Library/Frameworks/R.framework/Versions/4.1/Resources/library/cytolib/include/cytolib/GatingSet.hpp:14:
In file included from /Library/Frameworks/R.framework/Versions/4.1/Resources/library/cytolib/include/cytolib/GatingHierarchy.hpp:17:
In file included from /Library/Frameworks/R.framework/Versions/4.1/Resources/library/cytolib/include/cytolib/populationTree.hpp:14:
In file included from /Library/Frameworks/R.framework/Versions/4.1/Resources/library/BH/include/boost/graph/adjacency_list.hpp:255:
In file included from /Library/Frameworks/R.framework/Versions/4.1/Resources/library/BH/include/boost/graph/detail/adjacency_list.hpp:37:
In file included from /Library/Frameworks/R.framework/Versions/4.1/Resources/library/BH/include/boost/graph/adjacency_iterator.hpp:13:
/Library/Frameworks/R.framework/Versions/4.1/Resources/library/BH/include/boost/detail/iterator.hpp:13:1: warning: This header is deprecated. Use <iterator> instead. [-W#pragma-messages]
BOOST_HEADER_DEPRECATED("<iterator>")
^
/Library/Frameworks/R.framework/Versions/4.1/Resources/library/BH/include/boost/config/header_deprecated.hpp:23:37: note: expanded from macro 'BOOST_HEADER_DEPRECATED'
# define BOOST_HEADER_DEPRECATED(a) BOOST_PRAGMA_MESSAGE("This header is deprecated. Use " a " instead.")
^
/Library/Frameworks/R.framework/Versions/4.1/Resources/library/BH/include/boost/config/pragma_message.hpp:24:34: note: expanded from macro 'BOOST_PRAGMA_MESSAGE'
# define BOOST_PRAGMA_MESSAGE(x) _Pragma(BOOST_STRINGIZE(message(x)))
^
<scratch space>:12:2: note: expanded from here
message("This header is deprecated. Use " "<iterator>" " instead.")
^
In file included from RcppExports.cpp:4:
In file included from ./../inst/include/CytoML.h:4:
In file included from /Library/Frameworks/R.framework/Versions/4.1/Resources/library/flowWorkspace/include/flowWorkspace.h:6:
In file included from /Library/Frameworks/R.framework/Versions/4.1/Resources/library/flowWorkspace/include/flowWorkspace/pairVectorRcppWrap.h:15:
In file included from /Library/Frameworks/R.framework/Versions/4.1/Resources/library/RcppArmadillo/include/RcppArmadillo.h:34:
In file included from /Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include/Rcpp.h:57:
/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include/Rcpp/DataFrame.h:136:18: warning: unused variable 'data' [-Wunused-variable]
SEXP data = Parent::get__();
^
In file included from RcppExports.cpp:4:
In file included from ./../inst/include/CytoML.h:5:
In file included from ../inst/include/CytoML/openWorkspace.hpp:12:
../inst/include/CytoML/winFlowJoWorkspace.hpp:435:30: warning: '/*' within block comment [-Wcomment]
* "*[local-name()='edge']/*[local-name()='vertex']" is for ellipsoidGate
^
../inst/include/CytoML/winFlowJoWorkspace.hpp:866:18: error: no member named 'setDataOnlyFlag' in 'cytolib::scaleTrans'
curTran->setDataOnlyFlag(true);
~~~~~~~ ^
../inst/include/CytoML/winFlowJoWorkspace.hpp:882:16: error: no member named 'setDataOnlyFlag' in 'cytolib::scaleTrans'
curTran->setDataOnlyFlag(false);
~~~~~~~ ^
3 warnings and 2 errors generated.
make: *** [RcppExports.o] Error 1
ERROR: compilation failed for package ‘CytoML’
* removing ‘/Library/Frameworks/R.framework/Versions/4.1/Resources/library/CytoML’
* restoring previous ‘/Library/Frameworks/R.framework/Versions/4.1/Resources/library/CytoML’
Warning message:
In i.p(...) :
installation of package ‘/var/folders/_g/xp8hp7mn02z122v2092lzw499zps22/T//RtmpcPvBBy/file132e439dffd5e/CytoML_2.5.1.tar.gz’ had non-zero exit status
Here is additional information about my setup:
R version 4.1.0 (2021-05-18) -- "Camp Pontanezen"
Copyright (C) 2021 The R Foundation for Statistical Computing
Platform: x86_64-apple-darwin17.0 (64-bit)
BiocManager version 1.30.15
> tools:::.BioC_version_associated_with_R_version()
[1] ‘3.13’
Any help would be very appreciated. Thanks!
try to update cytolib and flowWorkspace from github first
I've updated cytolib and flowWorkspace directly from github and continue to get the same error
Input:
library(flowWorkspace)
library(CytoML)
setwd("/Users/user1/Desktop/Flow_Data_Import_Test/DummyProject_Folder/DummyExp_Diva_Export/DummyExp_Diva_Exp_Export")
flowData <- getwd()
gatingFile <- list.files(flowData, pattern=".xml",full = TRUE)
ws <- open_diva_xml(gatingFile)
ws
diva_get_sample_groups(ws)
gs <- diva_to_gatingset(ws)
sampleNames(gs)
gs_get_pop_paths(gs)
plotGate(gs[[1]])
Output:
> setwd("/Users/user1/Desktop/Flow_Data_Import_Test/DummyProject_Folder/DummyExp_Diva_Export/DummyExp_Diva_Exp_Export")
The working directory was changed to /Users/user1/Desktop/Flow_Data_Import_Test/DummyProject_Folder/DummyExp_Diva_Export/DummyExp_Diva_Exp_Export inside a notebook chunk. The working directory will be reset when the chunk is finished running. Use the knitr root.dir option in the setup chunk to change the working directory for notebook chunks.> flowData <- getwd()
>
> gatingFile <- list.files(flowData, pattern=".xml",full = TRUE)
> ws <- open_diva_xml(gatingFile)
> ws
Diva Workspace Version Version 9.1.2
File location: /Users/user1/Desktop/Flow_Data_Import_Test/DummyProject_Folder/DummyExp_Diva_Export/DummyExp_Diva_Exp_Export
File name: DummyExp.xml
Workspace is open.
Groups in Workspace
> diva_get_sample_groups(ws)
> gs <- diva_to_gatingset(ws)
Choose which group of samples to import:
1: DummySpecimen1
Selection: 1
Parsing 2 samples
loading data: /Users/user1/Desktop/Flow_Data_Import_Test/DummyProject_Folder/DummyExp_Diva_Export/DummyExp_Diva_Exp_Export/DummyExp_DummySpecimen1_DummyTube1.fcs
swap cols: FSC-H:FSC-W
swap cols: SSC-H:SSC-W
Compensating
Error in solve.default(comp) : 'a' (7 x 14) must be square
how about set xml_compensation_enabled = F?
Success!!
Wow what a relief. Thanks so much for your help!
Selection: 1
Parsing 2 samples
loading data: /Users/user1/Desktop/Flow_Data_Import_Test/DummyProject_Folder/DummyExp_Diva_Export/DummyExp_Diva_Exp_Export/DummyExp_DummySpecimen1_DummyTube1.fcs
swap cols: FSC-H:FSC-W
swap cols: SSC-H:SSC-W
Compensating
computing data range
transforming ...
parsing gates ...
loading data: /Users/user1/Desktop/Flow_Data_Import_Test/DummyProject_Folder/DummyExp_Diva_Export/DummyExp_Diva_Exp_Export/DummyExp_DummySpecimen1_DummyTube2.fcs
swap cols: FSC-H:FSC-W
swap cols: SSC-H:SSC-W
Compensating
computing data range
transforming ...
parsing gates ...
done!
> sampleNames(gs)
[1] "DummyExp_DummySpecimen1_DummyTube1.fcs"
[2] "DummyExp_DummySpecimen1_DummyTube2.fcs"
> gs_get_pop_paths(gs)
[1] "root" "/P1" "/P1/P2" "/P1/P2/P3" "/P1/P2/P3/P4"
[6] "/P1/P2/P3/P5" "/P1/P2/P3/P6" "/P1/P2/P3/P8"
Hi,
In cases where a compensation matrix was not created in Diva, diva_to_gatingset attempts to find one in the xml file and returns an error:
Error in solve.default(comp) : 'a' (2 x 4) must be square
.This makes importing xml gates from Diva impossible in many situations. Let me know if you require an example xlm / fcs set.