VForWaTer / metacatalog

Modular metadata management platform for environmental data.
https://vforwater.github.io/metacatalog
GNU General Public License v3.0
3 stars 1 forks source link

Add ResultSetList #140

Closed mmaelicke closed 2 years ago

mmaelicke commented 3 years ago

The export extension can export a single ResultSet. A new class is needed to wrap many ResultSets and drop everything that is duplicated, based on their md5 sums. This list can then iteratively export all contents into a tarball. Alternatively, some export endpoints might support merging.

mmaelicke commented 3 years ago

Additionally, the ListExporter needs an option to apply a unified (temporal) scaling by cutting extent and aggregate resolution. For upscaling: remove (data) from the merged dataset if the resolution does not match the needed target scaling.

For all variables, a hardcoded mapping is defined, which may be overwritten by Entry.details.get('metacatalog').get('aggregation function')`:

Sum: <7> daily rainfall sum, <19> evaptranspiration, <20> drainage Max: <13> precision Mean: everything else.

Implement this only optionally, as .harmonize_scale and a property to yield the excluded Entry due to scaling