FEATURE DEMO: generate gatingTemplate from parsed workspace for use with CytoExploreR

[Biomiha]

Hi @DillonHammill,

I've seen some similar issues on openCyto before but am not sure if that was ever resolved. I would like to export existing gates from a parsed diva workspace to a gating template for use with CytoExploreR. The initial part involves CytoML

ws <- open_diva_xml("20200422_myxmlfile.xml")
gs <- diva_to_gatingset(obj = ws, name = 3, subset = 1, worksheet = "normal")

This generates a gating set from the diva xml file and accompanying fcs files. That means the gating set has already got gates - mostly polygonal gates, whose vertices I would like to export to the gatingTemplate.csv akin to:

┌───────┬─────┬────────┬───────────┬───────────┬───────────┬───────────┬─────────┬───────────┬───────────┐
│ alias │ pop │ parent │ dims      │ gating_me │ gating_ar │ collapseD │ groupBy │ preproces │ preproces │
│       │     │        │           │ thod      │ gs        │ ataForGat │         │ sing_meth │ sing_args │
│       │     │        │           │           │           │ ing       │         │ od        │           │
├───────┼─────┼────────┼───────────┼───────────┼───────────┼───────────┼─────────┼───────────┼───────────┤
│ Beads │ +   │ root   │ FSC-A,SSC │ cyto_gate │ gate =    │ TRUE      │         │ pp_cyto_g │           │
│       │     │        │ -A        │ _draw     │ list('Com │           │         │ ate_draw  │           │
│       │     │        │           │           │ bined     │           │         │           │           │
│       │     │        │           │           │ Events' = │           │         │           │           │
│       │     │        │           │           │ new('filt │           │         │           │           │
│       │     │        │           │           │ ers',     │           │         │           │           │
│       │     │        │           │           │ .Data =   │           │         │           │           │
│       │     │        │           │           │ list(new( │           │         │           │           │
│       │     │        │           │           │ 'polygonG │           │         │           │           │
│       │     │        │           │           │ ate',     │           │         │           │           │
│       │     │        │           │           │ boundarie │           │         │           │           │
│       │     │        │           │           │ s         │           │         │           │           │
│       │     │        │           │           │ =         │           │         │           │           │
│       │     │        │           │           │ structure │           │         │           │           │
│       │     │        │           │           │ (c(232496 │           │         │           │           │
│       │     │        │           │           │ .19616042 │           │         │           │           │
│       │     │        │           │           │ ,         │           │         │           │           │
│       │     │        │           │           │ 216513.88 │           │         │           │           │
│       │     │        │           │           │ 062057,   │           │         │           │           │
│       │     │        │           │           │ 232496.19 │           │         │           │           │
│       │     │        │           │           │ 616042,   │           │         │           │           │
│       │     │        │           │           │ 296425.45 │           │         │           │           │
│       │     │        │           │           │ 8319821,  │           │         │           │           │
│       │     │        │           │           │ 344372.40 │           │         │           │           │
│       │     │        │           │           │ 4939371,  │           │         │           │           │
│       │     │        │           │           │ 348938.78 │           │         │           │           │
│       │     │        │           │           │ 0807899,  │           │         │           │           │
│       │     │        │           │           │ 332956.46 │           │         │           │           │
│       │     │        │           │           │ 5268049,  │           │         │           │           │
│       │     │        │           │           │ 357782.85 │           │         │           │           │
│       │     │        │           │           │ 8488803,  │           │         │           │           │
│       │     │        │           │           │ 229702.45 │           │         │           │           │
│       │     │        │           │           │ 9752674,  │           │         │           │           │
│       │     │        │           │           │ 185366.93 │           │         │           │           │
│       │     │        │           │           │ 7113244,  │           │         │           │           │
│       │     │        │           │           │ 168125.34 │           │         │           │           │
│       │     │        │           │           │ 4975688,  │           │         │           │           │
│       │     │        │           │           │ 185366.93 │           │         │           │           │
│       │     │        │           │           │ 7113244,  │           │         │           │           │
│       │     │        │           │           │ 249407.13 │           │         │           │           │
│       │     │        │           │           │ 6481309,  │           │         │           │           │
│       │     │        │           │           │ 370098.28 │           │         │           │           │
│       │     │        │           │           │ 14442     │           │         │           │           │
│       │     │        │           │           │ ), .Dim = │           │         │           │           │
│       │     │        │           │           │ c(7L,     │           │         │           │           │
│       │     │        │           │           │ 2L),      │           │         │           │           │
│       │     │        │           │           │ .Dimnames │           │         │           │           │
│       │     │        │           │           │ =         │           │         │           │           │
│       │     │        │           │           │ list(NULL │           │         │           │           │
│       │     │        │           │           │ ,         │           │         │           │           │
│       │     │        │           │           │ c('FSC-A' │           │         │           │           │
│       │     │        │           │           │ ,         │           │         │           │           │
│       │     │        │           │           │ 'SSC-A'   │           │         │           │           │
│       │     │        │           │           │ ))),      │           │         │           │           │
│       │     │        │           │           │ parameter │           │         │           │           │
│       │     │        │           │           │ s         │           │         │           │           │
│       │     │        │           │           │ =         │           │         │           │           │
│       │     │        │           │           │ new('para │           │         │           │           │
│       │     │        │           │           │ meters',  │           │         │           │           │
│       │     │        │           │           │ .Data =   │           │         │           │           │
│       │     │        │           │           │ list(new( │           │         │           │           │
│       │     │        │           │           │ 'unitytra │           │         │           │           │
│       │     │        │           │           │ nsform',  │           │         │           │           │
│       │     │        │           │           │ .Data =   │           │         │           │           │
│       │     │        │           │           │ function  │           │         │           │           │
│       │     │        │           │           │ () NULL,  │           │         │           │           │
│       │     │        │           │           │ parameter │           │         │           │           │
│       │     │        │           │           │ s         │           │         │           │           │
│       │     │        │           │           │ =         │           │         │           │           │
│       │     │        │           │           │ 'FSC-A',  │           │         │           │           │
│       │     │        │           │           │ transform │           │         │           │           │
│       │     │        │           │           │ ationId   │           │         │           │           │
│       │     │        │           │           │ =         │           │         │           │           │
│       │     │        │           │           │ 'defaultU │           │         │           │           │
│       │     │        │           │           │ nityTrans │           │         │           │           │
│       │     │        │           │           │ form'),   │           │         │           │           │
│       │     │        │           │           │ new('unit │           │         │           │           │
│       │     │        │           │           │ ytransfor │           │         │           │           │
│       │     │        │           │           │ m',       │           │         │           │           │
│       │     │        │           │           │ .Data =   │           │         │           │           │
│       │     │        │           │           │ function  │           │         │           │           │
│       │     │        │           │           │ () NULL,  │           │         │           │           │
│       │     │        │           │           │ parameter │           │         │           │           │
│       │     │        │           │           │ s         │           │         │           │           │
│       │     │        │           │           │ =         │           │         │           │           │
│       │     │        │           │           │ 'SSC-A',  │           │         │           │           │
│       │     │        │           │           │ transform │           │         │           │           │
│       │     │        │           │           │ ationId   │           │         │           │           │
│       │     │        │           │           │ =         │           │         │           │           │
│       │     │        │           │           │ 'defaultU │           │         │           │           │
│       │     │        │           │           │ nityTrans │           │         │           │           │
│       │     │        │           │           │ form'))), │           │         │           │           │
│       │     │        │           │           │ filterId  │           │         │           │           │
│       │     │        │           │           │ =         │           │         │           │           │
│       │     │        │           │           │ 'Beads')) │           │         │           │           │
│       │     │        │           │           │ )),       │           │         │           │           │
│       │     │        │           │           │ openCyto. │           │         │           │           │
│       │     │        │           │           │ minEvents │           │         │           │           │
│       │     │        │           │           │ = -1      │           │         │           │           │
└───────┴─────┴────────┴───────────┴───────────┴───────────┴───────────┴─────────┴───────────┴───────────┘

Column names: alias, pop, parent, dims, gating_method, gating_args, collapseDataForGating, groupBy, preprocessing_method,
preprocessing_args

I have been trying to figure out how to do this with capture.output(dput(openCyto::gs_pop_get_gate(gs[[1]], "P1")[[1]])) but it always looks formatted differently. If you could shed some light into how you export the gate info into the csv file, that would be absolutely brilliant. Many thanks.

[DillonHammill]

@Biomiha, this is on my list of things to do. Formatting the gates for the gatingTemplate is complex and it is not something that I would do manually. I am going to add a cyto_gatingTemplate_generate() function which will create a CytoExploreR gatingTemplate from a GatingSet. This should be straight forward if all samples share the same gates. The challenge will lie in determining how the samples were grouped if the gates are different.

[DillonHammill]

@Biomiha, I have just added cyto_gatingTemplate_generate() to obtain a CytoExploreR-ready gatingTemplate from an established GatingHierarchy or GatingSet. This should work fine for all gate objects (including quadrant gates) and boolean filters. Just supply the GatingHierarchy or GatingSet and specify a name for the gatingTemplate csv file. The gatingTemplate will be written to the csv file and returned as a gatingTemplate object.

# gs is a GatingSet
gt <- cyto_gatingTemplate_generate(gs,
                                   gatingTemplate = "gatingTemplate.csv")

It is likely that I may need to revisit this issue in the future, so I will put some notes here as a reminder:

Notes:

• Reference gates will be recorded as separate gates (i.e. will not be automatically updated when the original gate is altered). In order to appropriately handle reference gates we would require a "refGate" flag and/or a filterID with a minimal unique path to the original gate. The filterId would be required to construct the reference gate logic for the gating_args in the gatingTemplate. For example, a filterId of CD69+ applied to both CD4 T Cells and CD8 T Cells is not informative - CD4 T Cells/CD69+ is ideal. Even CytoExploreR does not do this, so it is unlikely to be present in the imported GatingHierarchy or GatingSet. I will update CytoExploreR to have more descriptive filterID's in case I decide to add support for this in the future.
• The GatingSet method simply uses the first GatingHierarchy to construct the gatingTemplate, this means that all samples MUST share the same gates. It may be possible to re-create grouping structure based on the extracted gates but this would require a lot of work (and I am not even sure if it possible).

Regardless, it should be enough to get you started @Biomiha. If you do have any reference gates you will need to manually modify their entries in gatingTemplate using cyto_gatingTemplate_edit() or just treat them as separate gates.

[Biomiha]

Many thanks @DillonHammill. The output itself looks exactly what I was hoping for. Oddly enough, however, I get an error when running on my imported gating set. The error I get is:

Error in .getFullPath(this_row[, parent], new_dt) : Not able to to find reference to: P1

P1 is the first population gated from root but I can still find it in the output gatingTemplate.csv. Looks like some kind of internal error.

[DillonHammill]

@Biomiha, are you able to share the output csv file and your GatingSet (privately) so I can troubleshoot this?

You can use cyto_save(gs[1], save_as = "GatingSet") to save the GatingSet to a folder that you can then share.

[Biomiha]

@DillonHammill unfortunately I'm not able to share this dataset but I'll see if I can replicate the error on some public data and share that.

[DillonHammill]

How about just the gatingTemplate? That should be all I need to see what is going on.

[DillonHammill]

@Biomiha, this should now be fixed! Please pull down the latest version of CytoExploreR and let me know how you go.

[Biomiha]

👍 Thanks @DillonHammill

[DillonHammill]

This means that you can now import any data using CytoML and analyze it with CytoExploreR. It will also make it easy to compare the results between different software and CytoExploreR.