search

unity-sds / unity-py

Apache License 2.0
2 stars 4 forks source link

STAC Collection Utils to support asset-role #69

Closed mike-gangl closed 5 months ago

mike-gangl commented 6 months ago

We currently use the asset type (data, metadata) as a key into the asset. This prevents us from having multiple 'data' types. We propose to use the uri/href of the dataFile as the key into the assets, and use the "metadata" and "data" types as asset-roles:

https://github.com/radiantearth/stac-spec/blob/master/item-spec/item-spec.md#asset-roles

Specifically, with regards to to/from_STAC:

https://github.com/unity-sds/unity-py/blob/main/unity_sds_client/resources/collection.py#L110-L113

Needs to be updated to use the asset filename as the key, the "type" as the asset-role.

https://github.com/unity-sds/unity-py/blob/main/unity_sds_client/resources/collection.py#L184-L191

the constructor for DataFile shouldn't use the asset_key as the key, but as the role. in fact, we'll probably want to update the name to asset_role to avoid any confusion. the asset key should be determined by the passed in href/uri.

mike-gangl commented 5 months ago

here's an example of a STAC item that U-DS expects:

https://github.com/unity-sds/unity-data-services/blob/dd2a49accc62a57c3589a050234e08ae4b517ba2/tests/cumulus_lambda_functions/cumulus_stac/test_item_transformer.py#L296-L348

mike-gangl commented 5 months ago

Can you provide an update on this ticket?

anilnatha commented 5 months ago

@mike-gangl I have a draft of this change on my local machine ready to test, and I want to test it using the working with stac tutorial notebook , but the file "/unity/ads/scratch/gangl/chirp/catalog.json" referenced in the notebook produces a 404. Is there another file I can reference to test this?

mike-gangl commented 5 months ago

That file (and the contents being referenced) is on the test jupyter hub instance. That file will not be correct for your new code as it doesn't know about asset-role. you'll need to update the local test files (https://github.com/unity-sds/unity-py/blob/develop/tests/test_files/SNDR.SS1330.CHIRP.20160829T2317.m06.g233.L1_AQ.std.v02_48.G.200425130422.json) to have the asset-role defined.

GodwinShen commented 5 months ago

@rtapella same question as the other related tickets, is this resolved by PR https://github.com/unity-sds/unity-py/pull/73?

anilnatha commented 5 months ago

@GodwinShen Yes. PR #73 also resolved this ticket.