search

openml / openml-python

Python module to interface with OpenML
https://openml.github.io/openml-python/main/
Other
276 stars 142 forks source link

Fix: update fetching a bucket from MinIO #1314

Closed eddiebergman closed 6 months ago

eddiebergman commented 6 months ago

Closes: #1304

Branch by @PGijsbers, just created a PR from it

Previously, each dataset had their own bucket: https://openml1.win.tue.nl/datasets61/dataset_61.pq

But we were advised to reduce the amount of buckets and favor hosting many objects in hierarchical structure, so we now have instead some prefixes to divide up the dataset objects into separate subdirectories:

https://openml1.win.tue.nl/datasets/0000/0061/dataset_61.pq

This commit has bypassed pre-commit. Tests should be updated too.

Reference Issue

What does this PR implement/fix? Explain your changes.

How should this PR be tested?

Any other comments?

eddiebergman commented 6 months ago

Hi @josvandervelde,

Just pinging this issue again. I reran the tests to see where it fails which you can find here: https://github.com/openml/openml-python/actions/runs/7527320791

If you need to run them manually, you can do so from Actions -> Test -> Run Workflow. Otherwise, committing to this branch will also trigger them :)

eddiebergman commented 6 months ago

They seemed to pass, I'm going to try create an empty commit to have it display here

codecov-commenter commented 6 months ago

Codecov Report

Attention: 3 lines in your changes are missing coverage. Please review.

Comparison is base (b06ecee) 38.26% compared to head (64df490) 84.19%. Report is 2 commits behind head on develop.

:exclamation: Current head 64df490 differs from pull request most recent head 96b72dc. Consider uploading reports for the commit 96b72dc to get more accurate results

Files Patch % Lines
openml/_api_calls.py 0.00% 3 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## develop #1314 +/- ## ============================================ + Coverage 38.26% 84.19% +45.93% ============================================ Files 38 38 Lines 5240 5239 -1 ============================================ + Hits 2005 4411 +2406 + Misses 3235 828 -2407 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

eddiebergman commented 6 months ago

Seems the only issue was uploading the code coverage with a server error, otherwise this seems all good now :)

PGijsbers commented 6 months ago

Hi @LennartPurucker you still have a blocking review here. are you ok with a merge?

LennartPurucker commented 6 months ago

Yup, sorry, this is fine to merge now. And we can do a release afterwards @eddiebergman