search

broadinstitute / pooled-cell-painting-profiling-template

:hammer_and_wrench: Use me to version control Pooled Cell Painting data and processing pipelines
BSD 3-Clause "New" or "Revised" License
2 stars 3 forks source link

handle erroring in comparment file checking #31

Closed ErinWeisbart closed 2 years ago

ErinWeisbart commented 2 years ago

During a recent run, there was what I believe to be a stochastic I/O error in process_configuration during 1./0.merge-single-cells that happened during # Assert all compartment files are present in each site folder before populating. This catches errors and tries again on the site it happens to.

It also returns a list of incomplete sites (those that fail compartment check) and a list of sites that errored during assertion.

ErinWeisbart commented 2 years ago

Additional information returned by function in handles in recipe by https://github.com/broadinstitute/pooled-cell-painting-profiling-recipe/commit/e8f72fbb30e48413b05e05a041d75ab12d6c8bca https://github.com/broadinstitute/pooled-cell-painting-profiling-recipe/commit/8afea284f808318fb345eda511be14cc7572a2fd