kharchenkolab / Baysor

Bayesian Segmentation of Spatial Transcriptomics Data
MIT License
150 stars 31 forks source link

Surprised at the poor result despite with accurate Prior Segmentation #131

Open tsuijenk opened 3 weeks ago

tsuijenk commented 3 weeks ago

Good day,

Below are the parameters I use for the prior segmentation:

n_clusters = 3 # Number of clusters to use for cell type segmentation. Default: 4
prior_segmentation_confidence = 0.96 # Confidence of the prior segmentation. Default: 0.2
iters = 4000 # Number of iterations for the cell segmentation algorithm. Default: 500
n_cells_init = 300 # Initial number of cells

XP

I thought I would be getting all the cell centers given I set the confidence to 0.96 and with such a higher number of iterations. Are there any other advice to improve the baysor result?

Thank you.

VPetukhov commented 3 weeks ago

@tsuijenk, can you please provide more information on how the results look now and how do you understand that they're poor?

tsuijenk commented 1 week ago

Hello! Thank you very much for your reply.

This image is the output of a CellPose trained model, where each nuclei is labeled by a unique integer, with background denoting 0. Hence, we would expect Baysor to be able to put a "red dot" on each of these nuclei. However, there are only three "red dots" here, far from expected outcome.

We have 16 FOVs and this was one of them. I wonder if the fact that them being tissue data made it more difficult. As Baysor seemed to have worked well on a cell cultured image set.