Prepare repository for production

broadinstitute / pooled-cell-painting-profiling-recipe

:woman_cook: Recipe repository for image-based profiling of Pooled Cell Painting experiments

BSD 3-Clause "New" or "Revised" License

6 stars 4 forks source link

Prepare repository for production #10

Closed gwaybio closed 4 years ago

gwaybio commented 4 years ago

We will need to rethink several decisions before the recipe is production-ready. We will use this issue to document these items.

Checklist

[ ] Determine where documentation lives (i.e. does it stay as READMEs or does it move to the wiki?)
[ ] Define clear user interaction via config or command line flags (likely means we'll migrate command line flags to config)
[ ] Indicate in documentation when a step touches a file made by another step so that user knows to update all appropriate config fields.
[ ] Add descriptive error messages when a step touches a file made by another step so that the user knows why the file is missing.
[ ] Rethink approach to populating Metadata_Foci_Cell_Category during 2.process-cells for when we move beyond simple cell quality categorizations.

ErinWeisbart commented 4 years ago

Where does documentation live? i.e. does it stay as READMEs or does it move to the wiki?

gwaybio commented 4 years ago

As @ErinWeisbart notes in https://github.com/broadinstitute/pooled-cell-painting-profiling-recipe/pull/8#issuecomment-631600540:

There are a few different points where there is the option to overwrite errors with --force. That's not coming from the config file, correct? Shouldn't those be extracted to the config file?

This is in reference specifically to 0.merge-single-cells.py but it is more general referring to all cases of user interaction in config vs. user interaction via command line flags

ErinWeisbart commented 4 years ago

In the config files, add documentation to every step that pulls files output by a previous step to indicate that the config must also be correctly for that output step.
e.g.
2.process-cells loads a .csv output by 1.process-spots When running 2.process-cells.py, it looks in the process-spots section of the config_utils.py to know where to find that output file.

Also add descriptive error messages to tell the user why a file doesn't exist (i.e. go run step X first)

ErinWeisbart commented 4 years ago

Currently in 0./2.process-cells we assign a cell quality category to un-assigned cells that is n+1 where n is the number of defined categories. This will probably need to change when we move to a more complicated and/or Bayesian cell category model. Discussed here.

gwaybio commented 4 years ago

this issue is too broad to be useful. Closing in favor of more succinct issues and issue tracking via milestones.