haniffalab / webatlas-pipeline

A data pipeline built in Nextflow to process spatial and single-cell experiment data for visualisation in WebAtlas
MIT License
46 stars 10 forks source link

Refactor #56

Closed dannda closed 1 year ago

dannda commented 1 year ago

Refactoring of the pipeline towards more general and extensible code. Major changes in main.nf. Slight changes in python scripts.

Modified the pipeline to have more general workflows Process_images and Process_files which handle data input mapping and call the appropriate processes. Removed technology-specific entrypoints.

Modified the tsv data input format as well, removing technology-specific headers in favor of specifying a file/image per line. This way arguments per file can be more easily specified if needed. Also included config_params as individual lines in the tsv file so it is easier to define each for a title-dataset group without having to duplicate that data in each line.

Ideally, this will allow us to easily add processing functions for new technologies/data formats.

codecov-commenter commented 1 year ago

Codecov Report

Base: 50.24% // Head: 49.76% // Decreases project coverage by -0.48% :warning:

Coverage data is based on head (df2d21a) compared to base (eae0273). Patch coverage: 42.85% of modified lines in pull request are covered.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## dev #56 +/- ## ========================================== - Coverage 50.24% 49.76% -0.48% ========================================== Files 10 10 Lines 416 428 +12 ========================================== + Hits 209 213 +4 - Misses 207 215 +8 ``` | [Impacted Files](https://codecov.io/gh/haniffalab/vitessce-pipeline/pull/56?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=haniffalab) | Coverage Δ | | |---|---|---| | [bin/build\_config.py](https://codecov.io/gh/haniffalab/vitessce-pipeline/pull/56?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=haniffalab#diff-YmluL2J1aWxkX2NvbmZpZy5weQ==) | `0.00% <0.00%> (ø)` | | | [bin/generate\_label.py](https://codecov.io/gh/haniffalab/vitessce-pipeline/pull/56?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=haniffalab#diff-YmluL2dlbmVyYXRlX2xhYmVsLnB5) | `80.00% <57.14%> (-14.12%)` | :arrow_down: | | [bin/process\_h5ad.py](https://codecov.io/gh/haniffalab/vitessce-pipeline/pull/56?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=haniffalab#diff-YmluL3Byb2Nlc3NfaDVhZC5weQ==) | `87.61% <100.00%> (ø)` | | | [bin/process\_molecules.py](https://codecov.io/gh/haniffalab/vitessce-pipeline/pull/56?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=haniffalab#diff-YmluL3Byb2Nlc3NfbW9sZWN1bGVzLnB5) | `77.77% <100.00%> (ø)` | | | [bin/router.py](https://codecov.io/gh/haniffalab/vitessce-pipeline/pull/56?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=haniffalab#diff-YmluL3JvdXRlci5weQ==) | `88.88% <100.00%> (ø)` | | Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=haniffalab). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=haniffalab)

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.