internetofwater / geoconnex.us

URI registry for https://geoconnex.us based URIs
Other
23 stars 14 forks source link

update reference feature contribution #177

Open ksonda opened 1 year ago

ksonda commented 1 year ago

The current CI/CD downloads everything (excepting specifically listed files) in the hydroshare resource https://www.hydroshare.org/resource/3295a17b4cc24d34bd6a5c5aaf753c50/ , then uses the github action to download them and convert them to tables within a PostGIS sql dump, which other CI/CD tools then use to update the production database.

We need to update both the process and guidance for reference feature contribution, since a Dockerfile no longer individually downloads gpkg files, whose names are automatically converted to postgres table names.

Theoretically, we can be agnostic to any OGR spatial data type, as long as

  1. the file is not corrupted
  2. the URL where it lives is public, live, and listed somewhere publicly in this repository
  3. the table name created by the github action corresponds to the table name in the pygeoapi.config.yml

I think the first choices to be made philosophically are:

  1. do we want all reference features to live in a common hydroshare resource, or are we fine with a registry to live URLs?
  2. If (1) is decided to be hydroshare resource, should we clean up that resource so only live files are there, and send the other files elsewhere
dblodgett-usgs commented 1 year ago

I think the question is really, do we need to make any changes to things now because the current set up is problematic? I think leaving them in a single hydroshare resource out of convenience and starting to point to individual file artifacts seems reasonable.

ksonda commented 1 year ago

In light of cgs-earth/hydrodump-action, which downloads everyhting in hydroshare resource for this CI/CD, we can consolidate the update process to placing new files in the hydroshare resource and updating the pygeoapi config, with a comment about update time if all that's happening is updating a file

webb-ben commented 5 months ago

Requires documentation published in github.com/internetofwater/reference.geoconnex.us