search

NOAA-OWP / wres

Code and scripts for the Water Resources Evaluation Service
Other
2 stars 1 forks source link

As a user, I want the WRES to output a sqlite artifact for my evaluations #260

Open epag opened 3 months ago

epag commented 3 months ago

Author Name: Chris (Chris) Original Redmine Issue: 92406, https://vlab.noaa.gov/redmine/issues/92406 Original Date: 2021-05-25

I was just looking at a ticket for the new @csv2@ schema and it struck me that the change could potentially be breaking if working with CSV since it's a semi-structured product, but that wouldn't really be a problem at all if it were output in a structured output, like a sqlite blob. A portable master blob would also be an incredibly helpful tool after the fact considering that many tools (like python + pandas) can read and manipulate it no problem.

spatialite would be preferable, but not always doable for generation. My laptop can read spatialite no problem, for example, but can't easily write it without recompiling stuff like python to look for a different sqlite3 executable (MUCH easier said than done).

I put COWRES in the subject since something outputting the sqlite would be neat - it just doesn't necessarily need to be the core WRES - and I'm setting this as low priority since it would be handy, but no one is actively clamoring for it.

Related issue(s): #163 Redmine related issue(s): 127159

epag commented 3 months ago

Original Redmine Comment Author Name: James (James) Original Date: 2024-02-01T13:13:31Z

Increasing the priority of this ticket. If we (continue to) accept that wres is the wrong tool for fancy graphics and custom evaluation products, then it becomes super important to facilitate the easy slicing/dicing/customization of our statistics datasets.

epag commented 3 months ago

Original Redmine Comment Author Name: Hank (Hank) Original Date: 2024-02-29T14:07:21Z

From Chris via chat:

I didn't know it at the time of writing, but geopackage is a type of sqlite database; imagine postgis, but for sqlite. THAT would be a great artifact if you could ship it with geometry because arc and qgis love geopackage.

Hank

epag commented 3 months ago

Original Redmine Comment Author Name: James (James) Original Date: 2024-02-29T14:09:34Z

Yes, good call, that's what we should aim for.