search

equinor / fmu-dataio

FMU data standard and data export with rich metadata in the FMU context
https://fmu-dataio.readthedocs.io/en/latest/
Apache License 2.0
10 stars 14 forks source link

Consistent storage/client usage of dates/datetimes stored in SUMO #389

Open jorgenherje opened 9 months ago

jorgenherje commented 9 months ago

Goal is to have a consistent way of handling dates/datetimes. As of now one can have a various formats.

Some data formats can be:

There should be a consistent format for storing/converting date/datetimes from SUMO to users?

Example:

Creating module in Webviz to show both summary data and observations in SimulationTimeSeriesMatrix-module. The summary data comes with numpy format and its timestamps_utc_ms format, whereas observations has a date string. By providing a string format through our api to front-end, the details and responsibility of date conversion is given to javascript. Ideally we would provide some common utility-functions which follows a uniform way of conversion rules to provide dates from back-end s.t. time zones etc are under control.

We could e.g. convert from date string, on specific format to timestamp_ms in our back-end, but it should be important to have certain expectations/criteria to the sting format from Sumo - e.g. iso 8601 format?

https://github.com/equinor/webviz/issues/469

perolavsvendsen commented 9 months ago

Storing of time stamps in local time without specifying time zone is also perhaps a discussion point here, possibly a separate issue (?)

jorgenherje commented 9 months ago

Storing of time stamps in local time without specifying time zone is also perhaps a discussion point here, possibly a separate issue (?)

I think it should be up for discussion whether it should be allowed to store time stamp without specifying time zone or not. If it ends up bein allowed, we should all agree on what fallback we should go for - so the users know what to expect. E.g. that the system adds T00:00:00Z for utc as fallback?

The conclusion of allowing time stamps without specifying time zone is probably a separate issue. But the outcome of that issue more or less decides which conversions are necessary. If we always can expect time zone, the work is easier. If not - we should agree on how to append "missing" details.

If such details are not handled - a lot of "auto magic" conversions might happen on the way from sumo to visualization in front-end.

perolavsvendsen commented 9 months ago

Sounds like time zone should always be included, at least for the timestamps in tracklog. Separate issue: https://github.com/equinor/fmu-dataio/issues/397

rwiker commented 9 months ago

I agree - as a minimum, the full timezone specification should be included, but I would prefer to see Zulu time used consistently. Conversion to local time for display should be handled by the client.

perolavsvendsen commented 8 months ago

Timestamps in the tracklog are now on UTC time (#397). However, there are also other time stamps related to the data (e.g. 4D survey dates, simulated time, etc) which is a slightly larger topic.