opendatacube / datacube-dataset-config

Scripts for indexing data into ODC instances
22 stars 27 forks source link

Updated for 9 class product #36

Closed SeffatChowdhury closed 2 years ago

SeffatChowdhury commented 2 years ago

@alexgleith please review

SeffatChowdhury commented 2 years ago

@fangfy Alex suggested to add the 9 class product def separately. Two differences: class 11 is used and two other classes are not used (Alex suggested to still keep them)

alexgleith commented 2 years ago

The way that stac-to-dc tool works, it takes the name from the Microsoft Planetary Computer STAC API Collection, so you need two products anyway.

There's no downside to having two product definitions, one for each of the products listed on the STAC API.

Regarding testing, you can create a local ODC instance and do it, or do it in the DE Africa DEV cluster, that's totally up to you!

fangfy commented 2 years ago

@fangfy Alex suggested to add the 9 class product def separately. Two differences: class 11 is used and two other classes are not used (Alex suggested to still keep them)

why are we keeping them if they are not used? if we keep unused classes then the two can be made exactly the same (except product name)

I think we should just have 9 class defined for the new one.

fangfy commented 2 years ago

The way that stac-to-dc tool works, it takes the name from the Microsoft Planetary Computer STAC API Collection, so you need two products anyway.

There's no downside to having two product definitions, one for each of the products listed on the STAC API.

Regarding testing, you can create a local ODC instance and do it, or do it in the DE Africa DEV cluster, that's totally up to you!

OK.... so PC is calling this second product io-lulc-9-class, which would only make sense if the other is called io-lulc-10-class. Do we have to keep this the same as PC? I just want us to have one io_lulc product in deafrica

alexgleith commented 2 years ago

No, it doesn't need to be named the same, but... you will need to change the indexing script to handle renaming it, which is a good idea.

This needs a new parameter added to it is all: https://github.com/opendatacube/odc-tools/blob/develop/apps/dc_tools/odc/apps/dc_tools/stac_api_to_dc.py

SeffatChowdhury commented 2 years ago

@alexgleith could you advise on how to contribute to repo odc-tools?

fangfy commented 2 years ago

No, it doesn't need to be named the same, but... you will need to change the indexing script to handle renaming it, which is a good idea.

This needs a new parameter added to it is all: https://github.com/opendatacube/odc-tools/blob/develop/apps/dc_tools/odc/apps/dc_tools/stac_api_to_dc.py

@SeffatChowdhury let me know if you'd like to try update this tool. I can review but I think you'll need to set up to be able to test the changes locally.

Regardless of the name though, the "flags_definition" need to match the actual product, so 9 labels listed for the 9 class lulc map.

SeffatChowdhury commented 2 years ago

@fangfy please assign me write access to odc-tools if possible, I can update it and will be able to test locally in Sandbox

fangfy commented 2 years ago

@fangfy please assign me write access to odc-tools if possible, I can update it and will be able to test locally in Sandbox

Change to this script will impact others, so you'll have to test it in a local odc installation before you can push and merge in the changes.

SeffatChowdhury commented 2 years ago

@fangfy that's ok, I do have a local ODC installation (non DEA) for testing

alexgleith commented 2 years ago

Hey @SeffatChowdhury you can create a pull request without needing write access to the repo. I'm happy to review the PR.

SeffatChowdhury commented 2 years ago

@fangfy please join this repo and review the PR https://github.com/opendatacube/odc-tools/pull/493 Thanks

SeffatChowdhury commented 2 years ago

@fangfy I'll revert to "not used", but as per @alexgleith review earlier, the list does need to be exhaustive