Open eric-czech opened 6 years ago
Recommendations from Allen Goodman (works on CellProfiler):
Datasets for benchmarking:
Notes from the CellProfiler team on other methods they've considered
They also mentioned that simulation tools like cytopaq (used to generate some datasets in Broad Biomage Benchmark datasets were not useful.
Relevant section from the CODEX preprint:
Software
Expanding on that list a bit:
Models Specific to Medical Imaging
Generic Architectures
Comments from @nsamusik on some things to keep in mind:
My main thought at this point is that the segmentation itself is just the first step, there also has to be a second step, where cell boundaries are optimized concomitantly with estimating the single-cell expression vectors. This way both the optimized cell boundaries and the expression data will likely look more accurate.
As for the benchmarking, I am happy to share a hand-labelled dataset that I have generated for the CODEX paper revisions. Here, each TIFF is matched with a TXT file that contains the coordinates of hand-labeled cell centers (X, Y, Z). There are no cell outlines labelled here, just the centers. In order to assess the segmentation quality, I computed several measures: R = Recall (% of hand-labelled centeres that ended up within a segmented cell region), S= Singlets (of those, what % how many ended up in a cell region with exactly 1 hand-labelled center), FPR = False positive rate (% cell regions without a hand-labelled center). Then I combined the three in a harmonic mean 3/(1/R + 1/S + 1/(1-FPR))
here's the link https://drive.google.com/open?id=1wUNaZ5dv2mDn_wwcSXlnfof6SwoQmlsq