search

kharchenkolab / Baysor

Bayesian Segmentation of Spatial Transcriptomics Data
https://kharchenkolab.github.io/Baysor/
MIT License
155 stars 31 forks source link

Over segmentation of cells in samples with non-uniform cell geometry #140

Open Riley-Grindle opened 1 month ago

Riley-Grindle commented 1 month ago

Hi Baysor Team,

I am a new-ish user to the baysor tool and have a series of Xenium samples our group is trying to re-segment. After multiple iterations of config settings it seems that I have yet to find an optimal segmentation profile that dynamically handles diverse cell shapes. With that said, is there a setting I may have missed, or a proper way to handle segmentation in cases of non-uniformity? I have attached my config.toml below. (Note: the prior segmentation being referenced was generated via XeniumRanger).

Additionally, it appears my z_location is being removed from associated output files.

Thank you for your consideration.

`[data] x = "x_location" y = "y_location" z = "z_location" gene = "feature_name" min_molecules_per_cell = 150

[segmentation] prior_segmentation_confidence = 0.2 scale = 15 scale_std = "100%" n_cells_init = 6456`

VPetukhov commented 1 month ago

Hi @Riley-Grindle ,

That's a good question. scale_std, unfortunately, doesn't work as intended and in my tests its effect on the results was minor (if any). I'd try setting num. clusters to 10-15 and increase scale to be closer to the size of larger cells. Also, if your prior segmentation doesn't work well (which I assume because of confidence 0.2), you can switch to nuclei segmentation and set a higher confidence. It would help with undersegmentation.

Additionally, it appears my z_location is being removed from associated output files.

What about other columns? x/y_location and feature_name?

Riley-Grindle commented 1 month ago

Disregard the last part of that query. The z_location is preserved in the transcript coordinates, and the cell/nuclei boundaries were never 3 dimensional to begin with.

Thank you for the suggestions! I will initiate a test run with some of these in mind and return with feedback.

VPetukhov commented 1 month ago

Let me know how that works! If that helps, I'll add it to the documentation.

Riley-Grindle commented 1 month ago

So I set the scale_std param to 0.8 and my initial scale to the largest observed cell radius (15) which seemed to allow the algorithm to handle highly variable sizes. The only thing i am now noticing is that there are cell boundaries that overlap slightly. Is this to be expected?

VPetukhov commented 1 month ago

Does that mean that the solution helped? :)

The only thing i am now noticing is that there are cell boundaries that overlap slightly. Is this to be expected?

Do you have 3D data? If so, it's often impossible to create non-overlapping polygons in 2D.