search

opengeospatial / ets-wfs20

Executable Test Suite for WFS 2.0
Other
9 stars 11 forks source link

AD Theme Direct WFS 2.0 Test - No Temporal values found #185

Closed robblis closed 1 year ago

robblis commented 3 years ago

To the OGC Community,

The following is the error:

OGC TEAM Engine reported a failed test: No temporal values found in results: property is "http://inspire.ec.europa.eu/schemas/ad/4.0 [inspire.ec.europa.eu]":beginLifespanVersion. expected [false] but found [true]

This is the decoded request that the validator sends

http://agsinspire1.esri.com/arcgis/rest/services/ADTest/MapServer/exts/InspireFeatureDownload/service?service=WFS&version=2.0.0&request=GetFeature&count=10&typenames=ns63:ThoroughfareName&namespaces=xmlns(xml,http://www.w3.org/XML/1998/namespace),xmlns(ns63,http://inspire.ec.europa.eu/schemas/ad/4.0),xmlns(wfs,http://www.opengis.net/wfs/2.0)&filter=tns:beginLifespanVersion</fes:ValueReference>1988-09-05T10:00:00Z</gml:beginPosition>1988-09-05T12:00:00Z</gml:endPosition></gml:TimePeriod></fes:During></fes:Filter>

The existing beginLifespanVersion value in the database for the ThoroughfareName entry is: 9/5/1988 6:00:00 AM If I adjust the request’s gml:beginPosition and gml:endPosition to the following highlighted below, by taking the “Z” off the time and adjusting the beginPosition time an hour before the current value, then I get the returned component.

http://agsinspire1.esri.com/arcgis/rest/services/ADTest/MapServer/exts/InspireFeatureDownload/service?service=WFS&version=2.0.0&request=GetFeature&count=10&typenames=ns63:ThoroughfareName&namespaces=xmlns(xml,http://www.w3.org/XML/1998/namespace),xmlns(ns63,http://inspire.ec.europa.eu/schemas/ad/4.0),xmlns(wfs,http://www.opengis.net/wfs/2.0)&filter=tns:beginLifespanVersion</fes:ValueReference>1988-09-05T05:00:00</gml:beginPosition>1988-09-05T12:00:00</gml:endPosition></gml:TimePeriod></fes:During></fes:Filter>

Our service returns datetimes without timezones. This means (as per XML Schema spec) the datetimes are in the servers local timezone. This timezone is unknown for external users, because it is not declared by the service.

I am not aware that the WFS spec supports declaring the timezone by the service.

Also, it almost seems like the Validator is assuming the datetimes returned by the service are in the local timezone of the validator.

Can someone please assess this issue and give us guidance?

Thank you.

dstenger commented 3 years ago

@robblis We transferred this issue to the ets-wfs20 issue tracker as it relates to the WFS 2.0 test suite.

dstenger commented 3 years ago

@keshav-nangare Can you please do further investigation?

keshavnangare commented 3 years ago

@robblis

Can you please provide service to analyze the reported behavior?

robblis commented 3 years ago

Hi @keshav-nangare,

The service for this test is running.

http://agsinspire1.esri.com/arcgis/rest/services/ADTest/MapServer/exts/InspireFeatureDownload/service?VERSION=2.0.0&SERVICE=WFS&REQUEST=GetCapabilities&LANGUAGE=&TYPENAMES=&MAXFEATURES=&COUNT=&f=image

Thanks, Robert

dstenger commented 3 years ago

W3C XML Schema Part 2 was checked (also see https://github.com/opengeospatial/ets-wfs20/issues/183#issuecomment-747364942) and states the following:

...
3.2.7.3 Timezones
...
The lexical representation of a timezone is a string of the form: (('+' | '-') hh ':' mm) | 'Z', where
...

Thus, the specification states that timezoned values are represented by a "Z" and that this is a legal notation.

@robblis So, a service should be capable to handle this notation. Does this answer your question and do you agree with the conclusion?

robblis commented 3 years ago

Hi @dstenger,

We finally were able to do some more testing on this.

So, we think the issue is the time conversion to "Z" time is not correct.

So, going back to the test stated above, the time query is: gml:beginPosition1988-09-05T10:00:00Z and gml:endPosition1988-09-05T12:00:00Z

The time for the beginLifespanVersion in the ThoroughfareName entry is: 9/5/1988 6:00:00 AM (PDT). This would translate to 13:00:00Z time. So I'm not sure why the generated query is generating the endPosition at 12:00:00Z. How is it determining that particular time?

image

If I run the same request and change the endPosition to13:01:00Z then a record is returned.

Is this a miscalculation for the time element, or does our time in the database need to be represented as "Z" time?

Thanks, Robert

dstenger commented 3 years ago

Hi @robblis,

Currently, your service is not reachable. Is it possible to enable it again so that we can use it for testing purposes?

If I understand you correctly, your service is supporting time patterns with "Z" but the calculation of the filter values is wrong? If that is the case, we will check the test suite regarding this behavior (a running test service would help us a lot).

Regards, Dirk

robblis commented 3 years ago

Hi @dstenger,

Which service are you trying to reach?

The service listed above... http://agsinspire1.esri.com/arcgis/rest/services/ADTest/MapServer/exts/InspireFeatureDownload/service?VERSION=2.0.0&SERVICE=WFS&REQUEST=GetCapabilities&LANGUAGE=&TYPENAMES=&MAXFEATURES=&COUNT=&f=image ...is reachable.

Thanks, Robert

robblis commented 3 years ago

Hi @ghobona,

Just wanted to see if there is any progress on this issue.

I verified the service in the link above is currently available.

Thanks, Robert

ghobona commented 2 years ago

Hi @robblis ,

The service is not reachable. Could you please check your firewall?

Regards,

Gobe

Screenshot 2021-12-01 at 10 30 44

JohannaOtt commented 2 years ago

@bpross-52n I have a similar issue with this WFS. grafik There are two kinds of error messages:

  1. No temporal values found in results: property is "http://inspire.ec.europa.eu/schemas/plu/4.0":validFrom. --> there is a validFrom value in the dataset in the SpatialPlan object (the OfficialDoument objects do not need one - see UML) and it is valid according to the INSPIRE Validator (dataset and report: validFrom.zip ) Could you please check why the error message is occurring?
  2. java.lang.AssertionError: t1 (Period: 2012-04-04T00:00:00.000+0000/2012-04-04T23:59:59.999+0000) is not related to t2 (Period: 2012-04-04T16:19:59.999+0000/2012-04-05T00:59:59.998+0000): BEFORE. The actual temporal relation is OVERLAPS -> not sure what t1 and t2 are in this case and what would be invalid about them.

If you want me to open a separate issue on that, please let me know. But as one of the error messages is the same as in this issue, I thought it would make sense to add it here.

Thank you and all the best

Johanna

bpross-52n commented 2 years ago

Thanks @JohannaOtt. I will look into this and get back to you.

JohannaOtt commented 2 years ago

@bpross-52n Please ignore number 1. That was fixed in this issue and does not happen on beta any longer. So it is only about no 2.

robblis commented 2 years ago

Hi @JohannaOtt,

We've turned on the following service to test against.

https://agsinspire1.esri.com/arcgis/rest/services/ADTest/MapServer/exts/InspireFeatureDownload/service?VERSION=2.0.0&SERVICE=WFS&REQUEST=GetCapabilities&LANGUAGE=&TYPENAMES=&MAXFEATURES=&COUNT=&f=image

Let us know if you need anything else from us.

Thanks, Robert

JohannaOtt commented 2 years ago

@robblis Not sure if that is a misunderstanding - I reported a similar issue as yours as a validator user. But I am sure your example service will be helpful for the validator team.

robblis commented 2 years ago

Hi @dstenger

Sorry, I had posted this also to #199 which is closed.

Wanted to check on this again and to add some more information that I'm still observing.

Testing the Direct WFS for the Sea Regions theme, I'm seeing generated in the "before Period" test the following GET request.

https://inspiredemo.esri.com/arcgis/rest/services/sr_test/MapServer/exts/InspireFeatureDownload/service?service=WFS&version=2.0.0&request=GetFeature&count=10&typenames=ns91:Sea&namespaces=xmlns(xml,http://www.w3.org/XML/1998/namespace),xmlns(ns91,http://inspire.ec.europa.eu/schemas/sr/4.0),xmlns(wfs,http://www.opengis.net/wfs/2.0)&filter=tns:beginLifespanVersion</fes:ValueReference>2017-07-12T17:40:00Z</gml:beginPosition>2017-07-12T18:20:00Z</gml:endPosition></gml:TimePeriod></fes:Before></fes:Filter>

If I remove the "Z" zulu reference in the DateTime value, the request returns a record.

https://inspiredemo.esri.com/arcgis/rest/services/sr_test/MapServer/exts/InspireFeatureDownload/service?service=WFS&version=2.0.0&request=GetFeature&count=10&typenames=ns91:Sea&namespaces=xmlns(xml,http://www.w3.org/XML/1998/namespace),xmlns(ns91,http://inspire.ec.europa.eu/schemas/sr/4.0),xmlns(wfs,http://www.opengis.net/wfs/2.0)&filter=tns:beginLifespanVersion</fes:ValueReference>2017-07-12T17:40:00</gml:beginPosition>2017-07-12T18:20:00</gml:endPosition></gml:TimePeriod></fes:Before></fes:Filter>

So it seems like the URL request generation from the validator is expecting all DateTime values to be in a Zulu format, which in our case, we can't do because the data for our services are in the local time zone of the server the service is running. on. I've attached the test log file.

Please let me know if you have any questions. We'll try to keep this service running so you can test.

Thanks, Robert

DirectWFS_Sea_Regions_s0036.zip

ghobona commented 2 years ago

Our interpretation is as follows.

The query above is using a temporal operator to compare time periods specified in the request with those provided in the data source.

Clause 7.9.3 of the Filter Encoding specification (OGC 09-026r1 / ISO 19143:2010) states that "**When temporal operators are used to compare time instants or periods, the time offset for local time zone shall be handled as described in XML Schema Part 2, 3.2.7: Datatypes (see W3C XML Schema Part 2)."

This implies that the server is expected to handle time zones when they are declared in a request.

SWG Chairs (@pvretano @cportele), is our interpretation correct?

cportele commented 2 years ago

@ghobona - Correct, if a time zone is specified in the predicate, the server has to take the timezone into account. Which is why the response of the server looks correct to me, although I have not analyzed this in detail.

The feature properties are:

Since the server does not return the timestamps with a timezone, i.e., the timestamps are in the local timezone of the server - in this case PDT. Now, if the client - in this case the CITE test - does not know the local timezone of the server, it cannot just assume that timestamps without a timezone are in UTC. The issue with the ETS is that it seems to make that assumption, which is invalid.

I think the only reasonable ETS approach would be to not use a timezone to filter temporal properties that are reported by a server without a time zone, because only then the test can assess, if the response is correct. If the server returns timestamps with a timezone, the ETS could also test proper timezone handling.

Here are some queries to demonstrate that the server seems to correctly handle timezone information:

ghobona commented 2 years ago

Thanks @cportele . That is a fair point.

@pvretanos, let us know if you have a different view.

mlcharv commented 2 years ago

Hi, We think we've run into this issue when validating this WFS, as explained on this report in the Validator Helpdesk: INSPIRE-MIF/helpdesk-validator#831 Just wanted to check if any progress has been made. Can we expect a fix in the near future? Thanks!

dstenger commented 2 years ago

@mlcharv We are currently working on the problem. Please check this issue for further updates.

bpross-52n commented 2 years ago

@bpross-52n I have a similar issue with this WFS. grafik There are two kinds of error messages:

  1. No temporal values found in results: property is "http://inspire.ec.europa.eu/schemas/plu/4.0":validFrom. --> there is a validFrom value in the dataset in the SpatialPlan object (the OfficialDoument objects do not need one - see UML) and it is valid according to the INSPIRE Validator (dataset and report: validFrom.zip ) Could you please check why the error message is occurring?
  2. java.lang.AssertionError: t1 (Period: 2012-04-04T00:00:00.000+0000/2012-04-04T23:59:59.999+0000) is not related to t2 (Period: 2012-04-04T16:19:59.999+0000/2012-04-05T00:59:59.998+0000): BEFORE. The actual temporal relation is OVERLAPS -> not sure what t1 and t2 are in this case and what would be invalid about them.

If you want me to open a separate issue on that, please let me know. But as one of the error messages is the same as in this issue, I thought it would make sense to add it here.

Thank you and all the best

Johanna

@JohannaOtt Regarding the first point: This is fixed on the beta instance of Teamengine.

Regarding the second point: This will be fixed by #241, see also here: comment