search

US-GHG-Center / ghgc-backend

Backend services for Green House Gas Center
Other
0 stars 1 forks source link

Fix/collection summaries utils #30

Closed anayeaye closed 10 months ago

anayeaye commented 10 months ago

Note This PR is a full copy of the PR with changes to solve the collection summaries problem in base veda-backend project which is 70 commits ahead of this fork. Only these changes are pulled in to support a quick fix. To cause existing databases to be updated, the VEDA_DB_SCHEMA_VERSION environment variable needs to be incremented before deployment.

What

This is an update to the collection summaries methods for two reasons:

  1. In the utility function dashboard.periodic_datetime_summary the datetime extent method was updated to extend to the maximum end_datetime from the collection's items (previously the nominal datetime was used which will not capture the full range of a collection of items that implement start/end datetimes).
  2. The cog_default summaries are removed to eliminate confusion--the method was only applicable to a very specific use case in which all items had an asset with the key cog_default that referred to a single band cloud optimized geotiff. The cog default summary is not used.
  3. The bulk update all collection default summaries method was simplified. This method selects all collections that use the dashboard:is_periodic property and then runs dashboard.update_default_collection_summaries for each.

Notes

  1. Two functions in the dashboard schema are changed but the function signature is not so the update can be applied to existing databases. The VEDA_DB_SCHEMA_VERSION must be incremented to cause the database handler to be run.
  2. The pgstac version was behind in docker compose so it is updated in this PR.

How tested

Tested with a temporary stack based on an older snapshot of the staging database. First the extent and datetime object the collection summaries was modified for one collection. Next pgstac.update_collection_extent(collection_id text) was tested and the result validated (this is important to know for a downstream ingest step that uses the method. Then the bulk dashboard.update_all_collection_default_summaries() method was executed to confirm that the periodic datetime summary utility function is working as well as the shortened bulk update all summaries function works as expected.

slesaad commented 10 months ago

I updated the VEDA_DB_SCHEMA_VERSION env var, should be good to merge!

Also changed the base branch to dev because that's the default for ghgc dev.

anayeaye commented 10 months ago

Thanks for swapping the base branch @slesaad!

anayeaye commented 10 months ago

Closing due to base branch problems, will create a new PR with only the database handler changes