DillonHammill / CytoExploreR

Interactive Cytometry Data Analysis
60 stars 13 forks source link

Confusing logicle exception during spill correction #129

Closed rwbaer closed 2 years ago

rwbaer commented 2 years ago

Briefly describe what you hope to achieve: I was attempting to do a spillover correction on 4 channel data. My transform step and spillover calculation seemed to work, and I see no errors. But when I open the matrix in the spillover editor I see a logicle exception that I can't interpret. If there was a problem with the transform, why don't I get an error when I apply the transform to the gating set. See image of editor with message..

Any help understanding what I'm doing wrong and how to fix it?

Outline the steps taken to attempt to reach this goal (paste code below):

Setup compensation controls

gsBeads <- cyto_setup(path = "Spill", gatingTemplate = "SpillTemplate.csv", restrict = TRUE)

Before going on, turn off unused channels in Viewer by deleting them in marker

column. Typically, everything but the FSC-HEIGHT, the area channel,

and time_lsw renamed to TIME.

Then click "Save & Close" button

Before going on, check the list of spill files inn viewer to see that it

is accurate. Click "Save and Close" button

Show us what we have

cyto_channels(gsBeads) cyto_markers(gsBeads) cyto_fluor_channels(gsBeads)

Transform fluorescent channels - default logicle transformations

gsBeads <- cyto_transform(gsBeads)

Gate beads

cyto_gate_draw(gsBeads, parent = "root", alias = "beads", channels = c("FSC-AREA", "SSC-AREA"))

Gate Single beads

cyto_gate_draw(gsBeads, parent = "beads", alias = "Single beads", channels = c("FSC-AREA", "FSC-HEIGHT"))

This next line is necessary if you are re-using gates from before

cyto_gatingTemplate_apply(gsBeads, "SpillTemplate")

Now compute spillover matrix

spill <- cyto_spillover_compute(gsBeads, parent = "Single beads", spillover = "Spillover-Matrix.csv")

Edit spillover matrix if so desired

spill <- cyto_spillover_edit(gsBeads, parent = "Single beads", spillover = "Spillover-Matrix.csv")


![spillError](https://user-images.githubusercontent.com/3155051/150040376-97d8fad6-6da6-4ac0-8eca-73d9118b4b5a.png)

**Include any associated screenshots or images here:**
DillonHammill commented 2 years ago

Looks like an issue with data transformations, hard to debug without a reproducible example. Perhaps one of your samples has no events in the parental gate?

A couple of suggestions:

  1. To avoid transformation errors it may be a good idea to switch to using type = "biex" in your cyto_transform() call. I would also set widthBasis = -100 if you decide to do this.
  2. Bypass the transformation step completely and let cyto_spillover_compute() handle this internally - it will resort to using biexponential transformations.
rwbaer commented 2 years ago

Thanks for the suggestions ... So, I think I now see that the error is arising during the plotting portion of the editor.

I will confess that the transformation aspects of analysis are a mystery to me despite having tried to read about it. I THINK is is about being able to visually apply better gates and that it does not affect data analysis itself, only visualization. If I understand correctly, transform objects are added to gating sets, but I'm not certain if the data itself is transformed, and how this is implemented for plotting or data summaries.

Anyway, I'm also attempting to attach my files so you might have a RepEx. 190122-Compensation-Channels.csv Spillover-Matrix.csv SpillTemplate.csv 180122-Compensation-Channels.csv 180122-Experiment-Markers.csv 180122-Experiment-Details.csv Spill.zip

DillonHammill commented 2 years ago

The data itself is stored in the transformed form but CytoExploreR takes care of reporting statistics (cyto_stst_compute()) on the original scale for you.

I am busy re-writing cyto_spillover_edit() for the new version of CytoExploreR so there is no point me debugging this version. I would recommend trying the new version when it is available and let me know if the issue persists.

rwbaer commented 2 years ago

As always thanks for the help.

DillonHammill commented 2 years ago

@rwbaer, had a quick look at the data you sent through and I have had some time to think about what might be going on here.

Firstly, what instrument are using to obtain the data? You data has a much larger dynamic range than standard flow cytometers. Since the current implementation of the logicle transform in CytoExploreR doesn't properly correct for this automatically (the new version does), I would recommend increasing width of the transformed display by increasing m above the default of 4.5.

gsBeads <- cyto_transform(gsBeads, type = "logicle", m = 8)

However, in my testing we still fail to correctly transform the data within in cyto_spillover_edit() in some cases. I have thought about this and I suspect that the problem is due to data becoming negative post compensation and therefore applying transformations defined on positive data only give rise to the error (because the transformation is not defined in this negative range). Basically looking at the internals it boils down the calculation of p (negative part of the scale) which in turn modifies w which must be strictly positive. I have played around with increasing a as well to display more negative log decades but we still run into the same issue with p. This may be more complicated than I had hoped but I will take a closer look when I get time - I may also need to rethink the way the data is transformed within cyto_spillover_edit() so that this works more reliably.

Secondly, I had a look at your spillover matrix and it appears as though the values are much higher that when I performed the automatic compensation on the same data. I am not sure why this is but perhaps you have set suboptimal gates when running cyto_spillover_compute() - I would recommend checking to make sure that the gates encompass most of the negative and positive events in your samples. Since you are using beads, I would avoid gating the tail of the distributions as this can skew things quite dramatically.

Anyways, I will revisit this issue when I have come up with a good way to resolve these transformation issues. In the meantime, I think you will need to use the biexponential transformation and use cyto_transformer_biex() to finetune the widthBasis until the data looks similar to the logicle transformed data. Again the default widthBasis -10 was set for standard cytometry datasests and as such you will need to optimise this yourself for your own data.