Error when calling normalize() or gs_clone() functions

[stefankolling]

Hi there,

As a heads-up, I've just started working with flowWorkspace. I'm trying to follow closely your example.R code made available for the HVTN080 trial, but I might be simply making some beginner's error.

When I try to normalize my gating set with a target sample, I keep getting the following error message produced by .cpp_CloneGatingSet() within the gs_clone() function:

cloning the gatingSet... cloning tree structure... Error in .cpp_CloneGatingSet(x@pointer, sampleNames(x)) : the current transformation is not found in global trans table!

I've tried to create an example with minimum code to illustrate the error:

library(knitr); library(parallel); library(gdata); library(flowWorkspace); library(ncdfFlow); library(data.table); library(Rgraphviz); library(flowViz); library(flowStats); library(CytoML); fcsPath<-"D:/Normalize/";

Parse workspace

ws <- open_flowjo_xml("D:/Normalize/gatingSet.xml"); try(G <- flowjo_to_gatingset(ws,name=4,subset=c('431345.fcs','431346.fcs'),useInternal=TRUE,additional.keys='', execute=TRUE,isNcdf=TRUE,path=fcsPath)); Parsing 2 samples mac version of flowJo workspace recognized. Creating ncdfFlowSet... All FCS files have the same following channels: Time FSC-A FSC-H SSC-A FITC-A PerCP Cy55 Blue-A Pacific Blue-A Am Cyan-A APC-A Alexa 680-A APC Cy7-A PE Green laser-A PE Tx RD-A PE Cy5-A PE Cy55-A PE Cy7-A done! loading data: D:/Normalize/431345.fcs Compensating gating ... write 431345.fcs to empty cdf slot... loading data: D:/Normalize/431346.fcs Compensating gating ... write 431346.fcs to empty cdf slot... done! flowjo_ws_close(ws); pop_to_norm <- "3+"; dim <- "3+"; target1 <- c("431345.fcs");

normalize: part of flowStats package; allows local normalization based on gating hierarchy

GN <- normalize(G,target=target1,populations=pop_to_norm,dims=dim); cloning the gatingSet... cloning tree structure... Error in .cpp_CloneGatingSet(x@pointer, sampleNames(x)) : the current transformation is not found in global trans table!

I run into the same problem when trying to clone a freshly parsed workspace:

library(knitr); library(parallel); library(gdata); library(flowWorkspace); library(ncdfFlow); library(data.table); library(Rgraphviz); library(flowViz); library(flowStats); library(CytoML); fcsPath<-"D:/MyData/My Universities/BSIO/RCode/CloneGatingSet/";

Parse workspace

ws <- open_flowjo_xml("D:/MyData/My Universities/BSIO/RCode/CloneGatingSet/gatingSet.xml"); try(G <- flowjo_to_gatingset(ws,name=4,subset=c('431345.fcs','431346.fcs'),useInternal=TRUE,additional.keys='', execute=TRUE,isNcdf=TRUE,path=fcsPath)); Parsing 2 samples mac version of flowJo workspace recognized. Creating ncdfFlowSet... All FCS files have the same following channels: Time FSC-A FSC-H SSC-A FITC-A PerCP Cy55 Blue-A Pacific Blue-A Am Cyan-A APC-A Alexa 680-A APC Cy7-A PE Green laser-A PE Tx RD-A PE Cy5-A PE Cy55-A PE Cy7-A done! loading data: D:/MyData/My Universities/BSIO/RCode/CloneGatingSet/431345.fcs Compensating gating ... write 431345.fcs to empty cdf slot... loading data: D:/MyData/My Universities/BSIO/RCode/CloneGatingSet/431346.fcs Compensating gating ... write 431346.fcs to empty cdf slot... done! flowjo_ws_close(ws); GN <- gs_clone(G); cloning tree structure... Error in .cpp_CloneGatingSet(x@pointer, sampleNames(x)) : the current transformation is not found in global trans table!

I suppose, I must be doing something fundamentally wrong, but fail to understand why the cloning fails for all code snippets I've come across and tried to run. Apologies for bothering you with this, normalization seems to work for everyone else on here.

I've attached my minimal code here: exampleNormalize.zip

Cheers, Stefan

[gfinak]

The normalization code was written years ago. It hasn't been supported for the current version of flowworkspace. With all the recent changes to the tooling, I'm not surprised that it is broken.

[stefankolling]

Thanks for the quick answer, then I stop wondering! I found your idea of local rather than global normalization interesting and just wanted to recreate it as a first step away from manual gating. So do I understand correctly that you stopped supporting the normalization code because there is no need for normalization before applying automated gating algorithms, such as openCyto? That's what it seems to me when looking at the Perforin example in your two papers from 2014 (normalization with template gating vs automated gating without normalization).

[mikejiang]

Since you didn't provide fcs file, I had to parse your xml by setting execute = FALSE, i.e.

> library(flowWorkspace);
> library(CytoML);
> ws <- open_flowjo_xml("~/Downloads/exampleNormalize/gatingSet.xml");
> G <- flowjo_to_gatingset(ws,name=4,subset=c('431345.fcs','431346.fcs'), execute=F);
> GN <- gs_clone(G);
> GN
A GatingSet with 2 samples
> packageVersion("CytoML") [1] ‘1.13.12’

As shown, the clone works just fine for the current Bioc devel branch. I will test the same on the current release (i.e. BioC 3.10 ) shortly once I setup the environment. In the meantime you can execute the same code to see if it succeeds as part of troubleshooting process.

[mikejiang]

I confirm that this error does occur in the current release. Since the devel branch (bioc 3.11) is going to become the release very soon, would you be able to upgrade your R to use Bioc 3.11 instead? I see the windows binaries are available for all our packages except flowWorkspace (which will also appear sometime today , thanks to the latest fix by @jacobpwagner ).

[gfinak]

That's correct. We tend not to normalize or use template gating.

Greg Finak

On Sat, Mar 28, 2020, 08:59 stefankolling notifications@github.com wrote:

Thanks for the quick answer, then I stop wondering! I found your idea of local rather than global normalization interesting and just wanted to recreate it as a first step away from manual gating. So do I understand correctly that you stopped supporting the normalization code because there is no need for normalization before applying automated gating algorithms, such as openCyto? That's what it seems to me when looking at the Perforin example in your two papers from 2014 (normalization with template gating vs automated gating without normalization).

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/RGLab/flowWorkspace/issues/312#issuecomment-605466487, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAKSI6N3FGKMEFVW3XDIJXLRJYNF7ANCNFSM4LVRRCWA .

[stefankolling]

Thanks for your help with troubleshooting! I've upgraded to Bioc 3.11 and now cloning and normalizing the gating set works again, just as you said.

I confirm that this error does occur in the current release. Since the devel branch (bioc 3.11) is going to become the release very soon, would you be able to upgrade your R to use Bioc 3.11 instead? I see the windows binaries are available for all our packages except flowWorkspace (which will also appear sometime today , thanks to the latest fix by @jacobpwagner ).

[stefankolling]

Thanks for the confirmation!

That's correct. We tend not to normalize or use template gating. Greg Finak