fraymio / modis-tools

Tools for working with the MODIS API and MODIS data.
Apache License 2.0
22 stars 11 forks source link

Fix pydantic for aws s3 urls #29

Closed XePeleato closed 1 year ago

XePeleato commented 1 year ago

When downloading from a collection such as MOD13A1 v061, the granule search will retrieve results hosted in an amazon s3 volume. Pydantic will not recognize such urls (s3://) and throw an exception: pydantic.error_wrappers.ValidationError: 8 validation errors for GranuleFeed

This commit removes the s3 urls from the query results.

Description

Closes # (issue)

Type of change

Please delete options that are not relevant.

How Has This Been Tested?

session = ModisSession(...) collection_client = CollectionApi(session=session)

col = collection_client.query(short_name='MOD13A1', version='061') granule_client = GranuleApi.from_collection(col[0], session=session) granules = granule_client.query( start_date=datetime(2023, 6, 8), end_date=datetime(2023, 6, 27), bounding_box=(5, 35, 5, 35) ) GranuleHandler.download_from_granules(granules, session, path='.')

which throws:
granule_feed = GranuleFeed(**feed)

File "pydantic\main.py", line 341, in pydantic.main.BaseModel.init pydantic.error_wrappers.ValidationError: 8 validation errors for GranuleFeed entry -> 0 -> links -> 1 -> href URL scheme not permitted (type=value_error.url.scheme; allowed_schemes={'http', 'https'})



## Checklist:

- [X] I have performed a self-review of my own code
- [X] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [X] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my feature works
- [X] New and existing unit tests pass locally with my changes
- [X] Any dependent changes have been merged and published in downstream modules
- [X] I have checked my code and corrected any misspellings

## Next Steps
- [ ] Assign a reviewer based on the [code owner document](https://github.com/fraymio/modis_tools/blob/main/.github/CODEOWNERS).

- [ ] Once your review is approved, merge and delete the feature branch

On behalf of the Modis Tools Dev Team, thank you for your hard work! ✨
XePeleato commented 1 year ago

Nevermind, just realized href being AnyUrl here. Wasn't like that in Pypi though, sorry about that!