Datacube-ows-serve CLI tool

whatnick commented 3 years ago

datacube-ows version: master ows_config.py version/link:: dynamic

Description

Wanted to preview some freshly produced products in an S3 bucket 🪣 via QGIS.

What I Did

Downloaded the data or used VSIS3 + a VRT to open it in QGIS and poke around.

What I would like to do

Have an OWS CLI tool called datacube-ows-serve which makes this a one-liner.

The CLI tool accepts the following parameters:

Blob store prefix for S3/Azure/GCP/Filesystem
Metadata YAML file or URL
Product YAML file or URL

It performs the following steps:

Bootstraps a Datacube with the given product and metadata using sqlite. ( Perhaps this should be 2 tools)
Indexes the data from storage into this embedded Datacube DB
Creates a smart set of default OWS styles and saves it to a JSON file for future tweaking
Fires up an embedded flask server to run an instance of OWS to serve the COGs from that location

Inspired by User Story from Erin Telfer.

SpacemanPaul commented 3 years ago

Cool idea, but I'm immediately struck by a few technical issues.

Seems to me that a large portion of this functionality doesn't really belong in OWS. Looks like it straddles the areas-of-responsibilities of multiple projects, including core, odc-tools and ows. It should probably be a part of new odc repo (odc-integration? odc-orchestration?) that depends on odc-tools and ows. OWS currently has no formal dependency on odc-tools and as currently spec'd this proposal would create a lot of clutter in the ows repo.
Will ODC core even work with a sqlite database? Does it have no dependencies on postgres-only SQL features? Even if core is OK, OWS definitely won't work off sqlite - no materialised views or postgis. Probably makes more sense to use the proposed new STAC interface Alex raised at yesterday's Steering Council meeting, and bypass the db entirely.
"Creates a smart set of default OWS styles" That's a really tall ask. A simple, dumb set of default OWS styles is a lot more realistic. :) There's also a fair bit more to OWS configuration than just styles, which would need to be dealt with.
All seems a bit Rube Goldberg. Surely there's a simpler way to meet this use case?

andrewdhicks commented 3 years ago

Could easier generation of a VRT be the answer instead? Have the tool produce the VRT, then it is just a matter of opening the file/URL in QGIS, and using QGIS's built in styles to view the data? Otherwise if it the data already in a datacube, then explorer could produce the VRT on the fly.

opendatacube / datacube-ows