search

dcjones / proseg

Probabilistic cell segmentation for in situ spatial transcriptomics
Other
45 stars 3 forks source link

How does Proseg use information of DAPI/Nuclear #36

Open wuwenrui555 opened 2 months ago

wuwenrui555 commented 2 months ago

Hi, thanks for your very useful tool!

I have read the preprint of the Proseg, and it says: "Cell morphologies are initialized using a nuclear stain, then expanded and altered at random until they best explain the observed data."

I am a little confused for this description, as the only input of the Proseg is a csv file containing information for all transcripts, where the images of DAPI are not involved. However, there is a column named "CellComp" (might generated from the result of Cellpose segmentation) contain information about the location of nucleus, which might be used as for cell morphologies initiation.

Does the Proseg use the location of transcripts with CellComp == "Nuclear" to initialize the cell morphologies? I wonder if the staining or the prior segmentation (which generates the CellComp column) is not good, will it also influence the result of Proseg?

Look forward to your kind reply! :D

dcjones commented 2 months ago

It does use the the CellComp column, rather than the DAPI stain directly. Those transcript assignments derived from the image-based segmentation are used in two ways:

  1. Voxel assignments are initialized (basically by assigning that voxel to whatever cell the majority of transcripts in that voxel are assigned to).
  2. Segmentation that disagrees with this initial segmentation is penalized.

It absolutely can lead to worse results if staining, or the initial segmentation based on the staining is poor quality. A potential remedy is to try to re-segment only nuclear with only the DAPI stain using something like Cellpose, which tends to be far more reliable than segmenting full cells using multiple channels.

Let me know if any of that's unclear.

wuwenrui555 commented 2 months ago

Very clear! Thanks for your detailed and kind reply!

And the idea that we can re-segment only nuclear with only the DAPI stain using whatever methods sounds pretty good, which makes the proseg method more flexible.

One more quick question: after I re-segment only nuclear with only the DAPI stain and get new assignments of transcripts, how should I modify the CellComp column? I suppose that I should assign all transcripts within the nuclear mask as "Nuclear" and assign the rest of transcripts as "None" or anything else.