opengeospatial / ogcapi-common

OGC API - Common provides those elements shared by most or all of the OGC API standards to ensure consistency across the family.
https://ogcapi.ogc.org/common
Other
45 stars 13 forks source link

Datetime: Add explicit examples of valid datetime parameters #342

Open jratike80 opened 3 weeks ago

jratike80 commented 3 weeks ago

As written now, the reader must conclude that because RFC 3339 applies, then the timezones must be supported in datetime. There is also a line in https://docs.ogc.org/DRAFTS/20-024.html#date-time-bnf-annex that supports this interpretation.

Note that unlike ISO 8601, the local time zone offset is required by RFC 3339.

I suggest to make the time zone offset requirement clear in the standard text. The right place might be https://docs.ogc.org/DRAFTS/20-024.html#datetime-parameter-requirements

Note that by RFC 3339 the datatime parameter must have either a "Z" suffix (1996-12-20T00:39:57Z) or an offset from the UTC time (1996-12-19T16:39:57-08:00). Datetimes without timezone ("local time" in ISO 8601) are not allowed.

A general set of requirements which all implementations must address should include the requirement to deal correctly with the offsets. And finally, some of the examples about datetimes should contain offsets. Now all of them are using the Zulu time.

Some other OGC standards (GeoPackage, JSON-FG) explicitly do not allow time zone offsets. It would reduce the confusion to emphasize that when a standard refers to common, then the support for time zones is compulsory.

jerstlouis commented 3 weeks ago

@jratike80

Some other OGC standards (GeoPackage, JSON-FG) explicitly do not allow time zone offsets. It would reduce the confusion to emphasize that when a standard refers to common, then the support for time zones is compulsory.

This is a good point, but I am somewhat confused about how we should address this.

The intent is that the Common requirement is as permissive as possible compared to any other OGC API standards that may want to declare a dependency on it.

The datetime requirement was originally taken from OGC API - Features - Part 1. Looking at the datetime parameter in Features, it simply says:

The syntax of date-time is specified by [RFC 3339, 5.6](https://www.rfc-editor.org/rfc/rfc3339.html#section-5.6).

I somehow assumed all this time that Features only requires the server to support understanding Z times.

There was a decision limit timezones to Z for CQL2 specifically: https://github.com/opengeospatial/ogcapi-features/issues/483 , so perhaps this is partially responsible for my confusion.

In fact, our implementation of Features was certified as passing all tests with only support for Z times.

However, it seems that you are right that RFC3339 allows support for specifying times in any time zones.

@cportele @pvretano

Is Features missing Abstract Tests (which guide the ETS development) to catch implementations like ours that are not implementing support for alternate time zones ?

Or should the requirements about datetime in Common and the various OGC APIs specifically only require support for Z, with a permission to understand client requests with datetime specified with other timezones?

In addition to Features and Common, we have the exact same datetime building block in several other OGC API standards, including EDR, Tiles, Maps, Coverages, DGGS and this should work consistently across all of them.

cportele commented 3 weeks ago

@jerstlouis

When we specified datetime, the goal was to align with RFC 3339 and the web practice, which meant to require a timezone. Since then we have moved to require UTC ("Z") to reduce requirements for timezone parsing (in CQL2 and JSON-FG).

If we would specify datetime today, I am sure it would also be constrained to UTC. Maybe we could add a recommendation in version 1.1 to use "Z".

In general, ATS tests are not super specific (CQL2 is different with its very detailed test cases). If we add tests for various kinds of timezones, we would also have to add tests for many other variations (e.g., years like "0001", nanoseconds, leap second cases, leap years, invalid values like "2024-02-31", etc.). Typically, this level of detail is added in the ETS. We can have a look at that in the version 1.1 revision, too.

jratike80 commented 3 weeks ago

Maybe we could add a recommendation in version 1.1 to use "Z".

How would developers read such recommendation? Probably like "Usually the times will come with Z, but also time offsets may appear, and you must handle them somehow".

I would like to see something that is explicit. Full RFC3339 support with Z and offsets is explicit. I believe that it is also possible to write an unambiguous definition for the UTC-only case. For example, we could stick with RFC3339, but define a profile that allows only Z or an offset of +00.00. We should also help the developers and tell what to do if the request still has some other offset, or comes without Z.

Could it be handled with requirement classes?

cportele commented 3 weeks ago

Implementations of Features Core that conform to the standard will always have to support time zones other than UTC. Everything else would be a breaking change.

The recommendation would target clients and recommend to use "Z", because that reduces the variations and all implementations likely will support "Z". It would not be a formal recommendation in the document as the standardization target are servers, not clients.

jratike80 commented 3 weeks ago

Implementations of Features Core that conform to the standard will always have to support time zones other than UTC. Everything else would be a breaking change.

Where in the OGCFeat Core that is written in an unambiguous way? Perhaps https://docs.ogc.org/is/17-069r4/17-069r4.html#_parameter_datetime where we can read a line "The syntax of date-time is specified by RFC 3339, 5.6." All the examples are using the UTC Z time, and there is not a single mention in the standard about time zones. Also for me it looks like the tests "Abstract Test 22" and "Abstract Test 23" do not test time zones.

Would it be a breaking change to the style we are using when writing the standards to say clearly what we mean? Like "A server SHALL support datetimes with time zone offsets. Server SHALL deny local times".

jerstlouis commented 3 weeks ago

@jratike80 I agree that clear requirements results into clear abstract tests which results in clear ETS which results in clearly conforming implementations. Now we have our implementation certified while it is actually not conforming correctly, and this same confusion has proliferated to other standards -- I will work on a fix for Maps, Coverages and DGGS at least.

What I would suggest for these and for Common is to say something like:

An implementation SHALL support date time as specified RFC 3339, 5.6 using the UTZ (Z) time zone.

And a permission that says:

An implementation MAY additionally support date time specified using time zone offset.

Regarding breaking change, I think what @cportele is saying is that it would be breaking clients to relax the requirement to support date time with time zone offsets.

It would not be a breaking change in a Features update to clearly state:

A server SHALL support datetimes with time zone offsets.

I really do believe Abstract Tests cases need to be added for the different cases that need to be supported.

Typically, this level of detail is added in the ETS

In practice, I don't believe this is really the case. It is quite a tedious and boring task to write an ETS, and developers usually go one Abstract Test at a time and would not want to spend more time on it than seems necessary. So the best way to ensure an exhaustive ETS and really ensure compliance achieves the desired level of interoperability, those ATS need to be exhaustive. We very much appreciated the level of details in the CQL2 tests, particularly the ones with tables of specific tests cases. This type of ATS level is gold for interoperability.

jerstlouis commented 1 week ago

@jratike80 We've clarified this with in Common with @joanma747 in Montreal last week, and those changes are also reflected in Maps, DGGS and Coverages:

https://github.com/opengeospatial/ogcapi-common/commit/f60b64976b0f39fa7b18be517c8679bc9503de51

What this says:

This should not break OGC API - Features, where the expectation is the servers always need to support local time offsets (although the ATS / ETS do not explicitly test for that).

jratike80 commented 1 week ago

This clarifies the matter. It leaves still some oddities especially into OGC API Features. The service must support local time offsets in parameter "datetime", but local time offsets cannot be in used in filter, at least when the query-lang is cql2, because CQL2 supports only UTC times.

How about the clients? Are they also required to support time offsets? If the client is requesting "2011-11-11T12:22:11Z", can the server return "2011-11-11T08:22:11-04:00" because by RFC 3339 they mean the same times? Or should the timezone in the response match with the timezone used in the query, if possible? That is not possible always because the outputformat may support only UTC, like in JSON-FG.

jerstlouis commented 1 week ago

@jratike80

How about the clients? Are they also required to support time offsets?

The target of the OGC APIs are the servers, so technically the clients can do whatever they want :)

In terms of the server response, there is the Content-Datetime: response header defined in OGC API - Maps (primarily because there are no fields to specify time in PNG or JPEG), and there are the response in the Features which the response payload format, which would be specified in the particular OGC API standard and/or format. (e.g., JSON-FG).