Haoran Chen and Robert F. Murphy and Ted Zhang\ Carnegie Mellon University\ V1.5 January 31, 2024
This package implements an approach for cell segmentation evaluation (CSE) that does not rely upon comparison to annotations from humans. For this, we defined a series of segmentation quality metrics that can be applied to multichannel images. The metrics are designed mainly for tissue images assuming that (1) there are multiple channels, and (2) there are multiple cell types that are expected to differ in their expression values for the channels. We calculated these metrics for 11 previously-described segmentation methods applied to 2D images from 4 multiplexed microscope modalities covering 5 tissues. Using principal component analysis to combine the metrics, we defined an overall cell segmentation quality score. The individual metrics and the quality score are returned.
The package supports both 2D and 3D images. It can be installed using
pip install CellSegmentationEvaluatorThen import the desired function, e.g.,
from CellSegmentationEvaluator import single_method_eval, single_method_eval3D, CSE3DThere are three main functions to evaluate an image and associated masks. For providing inputs as AICSImage structures, use
single_method_eval(imgpath, maskpath) #for 2D
single_method_eval3D(imgpath, maskpath) #for 3DOr for providing 3D images and masks as nd-arrays, use
CSE3D(img, mask, [PCAmodel], [threshimage], [voxel_size])where PCAmodel defaults to "3Dv1.6", threshimage defaults to the same of all channels, and the voxel size is in cubic micrometers and defaults to 1.
Reference:
Chen, Haoran, and Robert F. Murphy. "Evaluation of cell segmentation methods without reference segmentations." Molecular Biology of the Cell 34.6 (2023): ar50. https://doi.org/10.1091/mbc.E22-08-0364
Chen, Haoran, and Robert F. Murphy. "3DCellComposer: A versatile pipeline utilizing 2D cell segmentation methods for 3D cell segmentation" (2023)
This repository contains two implementations of the cell segmentation metrics as well as example images for testing. The first implementation ("SimpleCSE") calculates metrics and quality scores for one or more images and corresponding cell segmentation masks. The second ("full_pipeline") runs different cell segmentation programs on a given multichannel image and evaluates the resulting segmentations using the metrics. There is also a PyPI package described above.
It was tested on Python >=3.8 under Ubuntu 18.04 LTS.
Important: This repository uses Git LFS for tracking large files. Please ensure you have Git LFS installed on your machine before cloning or pulling.
Two input files are required, one containing a multichannel image (e.g., CODEX image) and the other containing the segmentation mask image to evaluate. The mask image should contain channels for a cell mask and a nucleus mask (in that order). Other mask channels may be present but will be ignored. Each channel should contain an indexed image, in which pixels contain the integer number of the cell that that pixel belongs to. Note that nuclear masks should not extend beyond cell masks; such nuclear masks will be truncated. Note also that cell and/or nuclear masks that are not matched will be ignored but reflected in the 'FractionOfMatchedCellsAndNuclei' metric.
This folder contains a simplified version of the CSE that calculates the metrics and quality score given a multichannel image and a corresponding cell mask.
It is provided as a Python main program and as an example Jupyter Notebook ("SegEvalExample.ipynb").
The multichannel image should be in a format readable by AICSimageio (e.g., OME-TIFF). The masks should be in a similar format with an indexed image for cell masks in the first channel and an indexed image (with corresponding indices) for nuclear masks in the second channel.
The output is a JSON file with the metrics and the scores.
Download the "SimpleCSE" folder and change your default directory ("cd SimpleCSE") to that folder
Run
pip install -r requirements.txt Run
python seg_eval_main.py [img_dir_path] [mask_dir_path]The "example_data" folder contains example 2D CODEX and 3D IMC images and their corresponding cell masks. To run them, use
python seg_eval_main.py example_data/imgs/2D_CODEX.ome.tiff example_data/masks/2D_CODEX.ome.tiffpython seg_eval_main.py example_data/imgs/3D_IMC.ome.tiff example_data/masks/3D_IMC.ome.tiffThis implementation can be used to seek the most suitable segmentation method for one or a batch of multichannel images.
Download "full_pipeline" folder and change your default directory to that folder
pip install -r requirements.txt python generate_config.py This step generates config.json which contains all configurational parameters and options in order to run the evaluation pipeline. The user will be asked a series of questions about how they would like to utilize this package. The answers will be stored in config.json for the pipeline to read. Meanwhile, all necessary data, software and dependencies will also be automatically downloaded or installed based on user responses.
python run_pipeline.py configuration_file_pathThis step runs the evaluation pipeline given the generated configuration file.
For a more detailed introduction to segmentation quality metrics among other image quality metrics, please see HuBMAP Image Quality Control Metrics
Reconcile differences with the 3D cell segmentation evaluation code in 3DCellComposer. Convert seg_eval_pkg.py to a PyPI package named CellSegmentationEvaluator and import that package for use in SimpleCSE. Add new function CSE3D as an alternative to "single_method_eval_3D" so that inputs can be nd-arrays rather than AICSImage structures.
Robert F. Murphy - murphy@cmu.edu\ Haoran Chen - hrchen@cmu.edu