pepkit / pipestat

Pipeline results reporting package
https://pep.databio.org/pipestat/
BSD 2-Clause "Simplified" License
4 stars 2 forks source link

Allow remote schemas. Fix #160 #162

Closed nsheff closed 6 months ago

nsheff commented 7 months ago

This allows the schemas to be remote. I did this by using yacman.load_yaml, instead of an internal read_yaml_data which did not handle URLs. I also replaced an internal mk_abs_via_cfg function call with ubiquerg.mkabs --

One nice thing about this is that it will make it so much easier to get started using pipestat. Our tutorials can just gloss over the schema complexity and point to some general-purposes, training schemas, so then the user doesn't have to create one locally to just get started quickly.

TODO:

donaldcampbelljr commented 7 months ago

Do you have an ETA on ubiquerg 0.8.0? Sometime in the next week?

nsheff commented 7 months ago

can be anytime. you can release it if it helps you. I don't have anything immediately planned to add to it.

codecov[bot] commented 6 months ago

Codecov Report

Attention: Patch coverage is 95.12195% with 2 lines in your changes are missing coverage. Please review.

Project coverage is 55.12%. Comparing base (4daf0c6) to head (a091ecb). Report is 6 commits behind head on dev.

:exclamation: Current head a091ecb differs from pull request most recent head 3e00e41. Consider uploading reports for the commit 3e00e41 to get more accurate results

Files Patch % Lines
pipestat/parsed_schema.py 94.11% 1 Missing :warning:
pipestat/pipestat.py 92.85% 1 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## dev #162 +/- ## ========================================== + Coverage 54.50% 55.12% +0.61% ========================================== Files 15 15 Lines 2341 2324 -17 ========================================== + Hits 1276 1281 +5 + Misses 1065 1043 -22 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

donaldcampbelljr commented 6 months ago

mk_abs_via_cfg contains the additional step of using os.makedirs to create the full path if it does not already exist. This is important for the file backend where the user could specify a results file that is nested into subfolders that don't exist yet. I added a commit that moved this functionality into a small separate func so that mkabs can be used everywhere and then we can create the subdirectories to the file path only when absolutely needed.

donaldcampbelljr commented 6 months ago

I've also refactored read_yaml_data with yacman.load_yaml.

I will go ahead and merge these changes to dev, adding the action item to update the documentation to the original issue in #160