ScalableCytometryImageProcessing / SCIP

Scalable Cytometry Image Processing (SCIP) is an open-source tool that implements an image processing pipeline on top of Dask, a distributed computing framework written in Python. SCIP performs projection, illumination correction, image segmentation and masking, and feature extraction.
https://scalable-cytometry-image-processing.readthedocs.io/en/latest/
GNU General Public License v3.0
7 stars 0 forks source link

Add pipeline and run configuration #3

Closed MaximLippeveld closed 3 years ago

MaximLippeveld commented 3 years ago

There is an important distinction between pipeline configuration and run configuration. The pipeline configuration concerns things like what channels to load, which segmentation algorithm to use on what channel, which features to compute... The run configuration concerns things like how many workers to use, how much memory each worker receives, what logging level to use...

Pipeline configuration should be implemented with a YAML config file that can be passed when launching the command line interface (CLI) on the command line. Example:

data_loading:
  file_format: multiframe_tiff
  channels:
    - 0
    - 1
    - 2
    - 4
    - 5
segmentation:
  noisy_channels:
    - 0
    - 5

The run configuration will be passed to the pogram through arguments of the CLI. Example

sip --n-workers 4 --worker-mem 20G

Argument handling is done with the click Python library.