Support for Webviz TimeSeriesPlugin

[perolavsvendsen]

Issue describing steps to undertake to support the TimeSeriesPlugin usecase in Webviz.

Requirements: SMRY-data/timeseries shall be supported by fmu-dataio, so that end users can use fmu-dataio for exporting such data with rich metadata according to the FMU standard. The metadata shall support the use case of migrating the TimeSeriesPlugin to use FMU results through Sumo.

Definition of Done: Necessary functionality is included in fmu-dataio to enable the TimeSeriesPlugin to work (either in current, or improved, state). TimeSeriesPlugin is functional on FMU results through Sumo.

Suggested first iteration feedback loop:

1. Start point is that SMRY data exists on disk during an FMU run. Establish a standalone Python script that 1) initializes fmu-dataio, 2) fetches the SMRY data from disk, 3) adds rich metadata, and 4) dumps data + metadata back to disk. Definition of done: It is possible, in a development setting, to produce SMRY-data with rich metadata on disk. Iteration with Webviz.
2. Building on feedback loop 1: Define what TimeSeries data should look like in the schema. E.g. define the data model, following the existing patterns in fmu-dataio/schema. Iteration with Webviz.
3. Add Sumo upload. Iteration with Webviz.

This does not necessarily happen in ordered fashion, but recommend to wait with 3 until certain degree of stability.

Identified subtasks (all are iterative):

• [ ] Metadata definitions for TimeSeries (SMRY-data) included in schema. We will keep to version tag 0.8.0 for now.
• [ ] Include tests in fmu-dataio that describes wanted functionality. Add more tests continuously, identifying corner-cases, etc.
• [ ] Functionality in fmu-dataio
• [ ] ERT config setup/good practice for export of SMRY-data
  • [ ] First version with standalone FORWARD_JOB sourcing a standalone Python that picks up SMRY-data from disk, initializes fmu-dataio, exports data to disk with metadata
  • [ ] Consider implementation in ecl2df or similar
  • [ ] Sumo upload
  • [ ] Webviz TimeSeriesPlugin rebuild (this is a huge task of course, not covered in this issue).

This issue relates to the larger task described here: https://github.com/equinor/webviz-subsurface/issues/1049

[perolavsvendsen]

Link to old code in first "proof of concept". The linked code was ran as a FORWARD_MODEL. It stopped pending functionality in fmu-dataio to take configuration/arguments from an external file (to avoid hard-coding in the job). (The interface for passing arguments to the FORWARD_JOB is too limited to ferry the necessary arguments from the config file.)

https://github.com/perolavsvendsen/webviz-subsurface/blob/06004fdcc05370a46518e3215229bcb07c73fc6e/webviz_subsurface/ert_jobs/smry2arrow.py#L157

[perolavsvendsen]

Offline discussions:

• Many small files, or one huge file? Split per vector? With one huge file, the associated metadata must be granular on vector-level inside the metadata structure. With many small files, granularity can be across the metadata files. Similarly, querying horizontally across multiple small files can be more difficult than across one row in one file.
  • Start off assuming 1 big file.
• Corner cases, e.g. reservoir coupling, multiple models.
• List all dates in the outgoing metadata? (index column)
• Robust for other sources, e.g. eCalc
• First step: Get something to run: "Drogon exports smry data with fmu-dataio, whatever they look like"

@rnyb @sago64