NERC-CEH / plankton_ml

A project for image processing and analysis pipelines for plankton sampling
GNU General Public License v3.0
0 stars 1 forks source link

Minimum change to reflect new bucket locations #18

Closed metazool closed 3 months ago

metazool commented 3 months ago

See #12 for context (changes to the layout of the object store).

This is a small and quite blunt change - i think any more work on making it generic should defer until

To test

pip install -e .
py.test 

The tests for new bucket locations will skip unless you have ENDPOINT and the JASMIN access tokens set in .env (they should skip in the pipeline but that's ok imo, just doing a slow file listing)

@Kzra

metazool commented 3 months ago

Considered adding documentation for the approach to intake here (expected to really complete #12) but better to keep PRs small and actually finish them :D

github-actions[bot] commented 3 months ago

β˜‚οΈ Python Coverage

current status: βœ…

Overall Coverage

Lines Covered Coverage Threshold Status
123 105 85% 0% 🟒

New Files

File Coverage Status
cyto_ml/tests/test_object_store.py 60% 🟒
TOTAL 60% 🟒

Modified Files

File Coverage Status
cyto_ml/data/s3.py 64% 🟒
cyto_ml/tests/conftest.py 100% 🟒
TOTAL 82% 🟒

updated for commit: 73d5c1e by action🐍

metazool commented 3 months ago

Updated to fix a test failure where .env is present but with arbitrary string instead of ENDPOINT location

Should bring this back in sync with the state of cyto-ML project @Kzra - are you happy to add an approving review? Please also see an issue there about preserving more of the metadata during the decollaging process :D

metazool commented 3 months ago

Much appreciated, @albags !