STAC Items API - Githubissues

datakaveri / ogc-resource-server

OGC compliant IUDX based resource server serving Geospatial data

Apache License 2.0

0 stars 6 forks source link

STAC Items API #183

Open mahimatics opened 3 months ago

mahimatics commented 3 months ago

Adding best practice fields Test using CARTOSAT-3 metadata and assets

mahimatics commented 3 months ago

Sandeep and Akhil have had initial discussions and we have come to a working consensus about how to model CARTOSAT or other similar datasets. Akhil needs to work on the generation tool to test out the implementation of STAC Items including /items, /search. Additionally we may need to add extra STAC extensions such as Transactions, CQL, EO, Grid etc.

Akhil also to explore Rakshit's suggestion about what to do regarding existing STAC entries for vector datasets. There are 4 options:

Do nothing
Add a dummy STAC item under each collection which will have the full asset file associated
Remove the STAC collection entries for vector datasets and explore using just the OGC Features
Remove the STAC collection entries for vector datasets and explore using something new like OGC Records

mahimatics commented 3 months ago

Open question: Is it possible for the properties objects of two STAC items within the same collection to adhere to different JSON schemas?

mahimatics commented 2 months ago

In the properties object, we have to decide which fields needs to be indexed for search API as queryables. Sandeep needs to be consulted for more information about which fields are important for CARTOSAT. Since this is resource specific, this has broader implications to the STAC onboarding flows.

mahimatics commented 2 months ago

Provider information, licence, and band information are going to be queryables for CARTOSAT. Metadata can also be included in the assets, however that will not be searchable. Sandeep's input is needed for deciding what metadata goes where.

From the API perspective, this may not be important. Work on /search is in progress. /items is complete.

ThorodanBrom commented 2 months ago

Added Flyway migration for STAC Items tables
Will be adding more commits to PR later on

mahimatics commented 2 months ago

Item search API testing is in progress Extra extensions for Sentinel required Sentinel Item templates to be shared Ingesting the assets is yet to be completed Item ID is yet to be decided based on the GIS usecase

mahimatics commented 1 month ago

NRSC data to be onboarded. Points to be decided

Items template
Items ID
Item count
Band information

mahimatics commented 1 month ago

https://docs.google.com/spreadsheets/d/1HvMHycZ_-WtOM0EqD0iZvIA3OAPW1p2JdV6p9A2r-V4/edit?usp=sharing

Item ID and metadata decided. STAC extensions can be generated with the metadata. Merged bands as a generated asset is good, but are they feasible within the timeline?

District name to be included as a generated attribute in the properties

mahimatics commented 1 month ago

Data has been uploaded and extracted into a new EC2 instance with a massive EBS volume. Total unzipped data size is 583G.

Varanasi have a few zip files which are broken (around 9 of them). They have been noted.

Symlinks resolution works in s3sync. So the script has been modified to create all the symlinks.

The script needs to be tested a few times and then run on the production instance.