jluethi
commented
5 months ago Only exposing the options does not deal with the sparsity issue we were having. In my tests, setting normalize = False with our existing (underexposed) 16bit images just leads to Cellpose not finding any objects.
I suspect we'd really have to implement rescaling of the data outside of cellpose to make this work. Will take a bit of experimentation on whether it makes sense to pass float32 normalized datasets to cellpose or not. And if that's the correct way, would need to think about the best way to expose this in the task
Some users need advanced options when running Cellpose tasks. We should expose more of them (& figure out how to separate between regular options and advanced options).
A few options that have come up that would be good to expose: Normalization/Quantile rescaling. This leads to issues in sparse images when not turned off otherwise.
Main parameters I'd like to expose:
bool (optional, default True) normalize data so 0.0=1st percentile and 1.0=99th percentile of image intensities in each channelOther parameters that we could also expose, just to have all options available:
int (optional, default 8) number of 224x224 patches to run simultaneously on the GPU (can make smaller or bigger depending on GPU memory usage)bool (optional, default False) invert image pixel intensity before running network (if True, image is also normalized)=> does not hurt to exposetile: bool (optional, default True) tiles image to ensure GPU/CPU memory usage limited (recommended)=> not hurt in exposing thistile_overlap: float (optional, default 0.1) fraction of overlap of tiles when computing flows=> no hurt in exposing thisresample: bool (optional, default True) run dynamics at original image size (will be slower but create more accurate boundaries)=> no hurt in exposing thisinterp: bool (optional, default True) interpolate during 2D dynamics (not available in 3D) (in previous versions it was False)=> no hurt in exposing thisstitch_threshold: float (optional, default 0.0) if stitch_threshold>0.0 and not do_3D and equal image sizes, masks are stitched in 3D to return volume segmentationParameters we shouldn't currently expose:
int (optional, default None) if None, z dimension is attempted to be automatically determinedrescale: float (optional, default None) if diameter is set to None, and rescale is not None, then rescale is used instead of diameter for resizing image: We enforce a diameter being passed, thus this option is not relevantOpen questions Can we provide other rescaling strategies? One idea is letting the user provide the rescaling quantiles (e.g. so they could set them to 0.0, 0.99999). The Cellpose API doesn't seem to provide one. One approach would be that Fractal itself performs some form of rescaling before passing the image to the Cellpose API. Cellpose rescales the image to be a float32 between 0 & 1. The open questions would be: What statistics are used to determine this rescaling values. Fixed-values, quantiles or statistics of a whole dataset? (cc @fstur for discussions we've been having on this recently)
All these options are in the model.eval call: https://github.com/MouseLand/cellpose/blob/4f5661983c3787efa443bbbd3f60256f4fd8bf53/cellpose/models.py#L109
And would need to be added here in Fractal: https://github.com/fractal-analytics-platform/fractal-tasks-core/blob/e5b8a0cefba076c3b6a0f3a1d0ce8909b4547d5c/fractal_tasks_core/tasks/cellpose_segmentation.py#L116