INSPIRE-MIF / helpdesk-validator

Community discussion forum for INSPIRE validation issues
42 stars 23 forks source link

Validator complains schemaLocation is not defined at root element #398

Open fhoubie opened 4 years ago

fhoubie commented 4 years ago

Using validation for Download Service Direct WFS, one failure is due to missing schemaLocation. The schemaLocation is present in the XML, but at the Element level, not root.

The schema attribute 'xsi:schemaLocation' was not found in the XML root element of the response. The response cannot be validated.

According to XML specification here, it says:

  1. xsi:schemaLocation and xsi:noNamespaceSchemaLocation [attributes] can occur on any element. However, it is an error if such an attribute occurs after the first appearance of an element or attribute information item within an element information item initially ·validated· whose [namespace name] it addresses. According to the rules of Layer 1: Summary of the Schema-validity Assessment Core (§4.1), the corresponding schema may be lazily assembled, but is otherwise stable throughout ·assessment·. Although schema location attributes can occur on any element, and can be processed incrementally as discovered, their effect is essentially global to the ·assessment·. Definitions and declarations remain in effect beyond the scope of the element on which the binding is declared.

Test service : http://sdidemo.intergraph.pl/WFS_INSPIRE/service.svc/get?ACCEPTVERSIONS=2.0.0&REQUEST=GetCapabilities&SERVICE=WFS&VERSION=2.0.0 Test run log: https://inspire.ec.europa.eu/validator/v2/TestRuns/EID246e2062-8e6c-465b-9fdd-fc7fde3896a1.html

carlospzurita commented 4 years ago

Dear @fhoubie

Thank you for opening this issue and providing the service endpoint. We are reviewing the ETS and the TG for Download Services, and will get back to you with an answer.

fhoubie commented 4 years ago

Thanks @carlospzurita , I've updated my comment as it was not clear where the issue was.

iuriemaxim commented 3 years ago

Dear @fhoubie

According to INSPIRE TGs, the file should be encoded according to GML 3.2, rules, not according to more general XML rules, therefore the OGC GML 3.2 standard should be followed. Indeed GML files are based on XML, but they follow some different rules.

Please check the https://www.ogc.org/standards/gml, or more precisely http://portal.opengeospatial.org/files/?artifact_id=20509 at page 255:

image

Therefore the test is imposed by OGC, not by INSPIRE.

@carlospzurita It can be marked as discussion

fhoubie commented 3 years ago

Dear @iuriemaxim , In the present case, it is the response to the GetCapabilities operation, which is not a GML application schema but a standard XML. The schemalocation deals with the INSPIRE extended capabilities element.

iuriemaxim commented 3 years ago

@fhoubie Sorry for confusion but was not clear for me that the problem was in relation to a GetCapabilities document. In any case, the situation is similar as the requirement for GetCapabilities content is also from OGC, not from XML. If in doubt, question can be raised to OGC. The OGC WFS 2.0,2 standard can be consulted at http://docs.opengeospatial.org/is/09-025r2/09-025r2.html

image Same is for Basic WFS and for WFS that implements all conformance classes.

Section 7.4 from OGC 06-121r3:2009 can be consulted as well. image

fhoubie commented 3 years ago

Dear @iuriemaxim ,

thanks, the specifcation states:

7.8 Use of the schemaLocation attribute Any GML document generated by a WFS shall reference an appropriate GML application schema document so that the output can be validated. This shall be accomplished using the schemaLocation attribute, as defined in the XML Schema specification (see W3C XML Schema Part 1).

EXAMPLE The following XML fragment shows the use of the schemaLocation attribute on the root element indicating the location of the an XML Schema document that can be used for validation:

<?xml version="1.0" ?> <wfs:FeatureCollection xmlns="http://www.someserver.example.com/myns" xmlns:myns="http://www.someserver.example.com/myns" xmlns:gml="http://www.opengis.net/gml/3.2" xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance xsi:schemaLocation="http://www.someserver.example.com/myns http://www.someserver.example.com/wfs.cgi?request=DescribeFeatureType&amp;typenames=TreesA_1M,RoadL_1M"> …

In this example, the service uses a DescribeFeatureType operation (see Clause 9) to call back to itself to reference a schema document where the TreesA_1M and RoadL_1M elements are declared.

So, it says "as defined in the XML Schema specification". Then, it shows an example of using schemaLocation on the root, not that it is mandatory to use it at the root. I agree for a GML Application schema as defined in GML specification, because the result is a FeatureCollection of a specific type, but not for the GetCapabilities operation.

iuriemaxim commented 3 years ago

Dear @fhoubie For GML it is clearly stated in the ATS A.3.1. You may ask OGC why they have an ATS if according to you the GML standard is not so clear and does not impose such a requirement that is only provided as an example. I am reading the examples as being part of the standard as well. I am reading the text "EXAMPLE The following XML fragment shows the use of the schemaLocation attribute on the root element indicating the location of the an XML Schema document that can be used for validation:" as being the reason of the ATS A.3.1. test, ilustrating that examples are part of the standard.

As I mentioned it can be read also the OGC 06-121r3:2009 where are provided a lot of examples for responses, including for GetCapabilities (section 7.4) as the WFS standard is indicating it in order to clarify how the GetCapabilities response should be encoded.

If the OGC 06-121r3:2009 is not clear, probably the only solution remains to clarify with OGC, not with INSPIRE.

iuriemaxim commented 3 years ago

Even most probably is not relevant for clarifications, but the OGC standards mentioned above are indicating the XML version from 2 May 2001 http://www.w3.org/TR/2001/REC-xmlschema-1-20010502/ and not the one from 2004 indicated in the first post https://www.w3.org/TR/2004/REC-xmlschema-1-20041028/

image

fhoubie commented 3 years ago

I don't think there is a link. There are 2 topics here: 1) mandate schemaLocation on the root of a GML response containing Features 2) mandate schemaLocation on the root of a GetCapabilities response to allow schema validation.

I'm raising the point on item n°2. There is no such requirement on the WFS 2.0 ATS. It is just written that it shall be possible to validate the XML.

In the INSPIRE test named "Validate Capabilites response against xsi:schemaLocation", the logic used to validate the XML is not correct according to me as it does not base itself on the XML specification, but on the specific presence of schemaLocation on a specific element, probably to make validation easier, but in practice, it is not correct as neither XML nor OGC WFS ATS mandates schemaLocation at the root for the Get Capabilities response document. So, I can't submit a bug to OGC as this test is done specifically in INSPIRE validation test suite.

Extract from ds-wfs-direct-soapui-project.xml


<con:config method="GET" name="GetCapabilities" timeout="0" xsi:type="con:HttpRequest" id="4850c8d2-28de-499e-82a8-7bf221f0ea14"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <con:description>Implementation note: The GetCapabilities request includes both the "AcceptVersions" and the "version" parameters. Strictly, for WFS 2.0 / OWS Common 1.1, only the "AcceptVersions" parameter is specified. However, for backward compatibility, the "version" parameter, which has been used in earlier versions of the WFS standard, is still included as a deprecated parameter in OWS Common 1.1. OWS Common 1.1.0 states: "A server may also optionally implement the old-style version negotiation mechanism so that old clients that send GetCapabilities requests containing a 'version' parameter can be served." To cover both the old and the newer version negotiation approach, the request includes both parameters. User specified parameters are dropped in this request.</con:description>
    <con:settings>
        <con:setting id="com.eviware.soapui.impl.wsdl.WsdlRequest@request-headers">&lt;xml-fragment/></con:setting>
</con:settings>
<con:encoding>UTF-8</con:encoding>
<con:endpoint>${#Project#endpt.GetCapabilities.Get}</con:endpoint>
<con:request/>
<con:assertion type="OwsExceptionReportAssertion" name="Expect no exception" id="e5ffedd2-dbba-45ff-9fc7-0ff804501a51"/>
<con:assertion type="Valid HTTP Status Codes" id="e11365e2-a2d0-4c80-8831-9880ea455af5" name="HTTP status code 200 is returned">
    <con:configuration>
        <codes>200</codes>
    </con:configuration>
</con:assertion>
<con:assertion type="Response SLA Assertion" id="8882b4e8-2fcd-4716-92f3-cd4d9f4d3615" name="30 seconds timeout">
    <con:configuration>
        <SLA>30000</SLA>
    </con:configuration>
</con:assertion>
<con:assertion type="GroovyScriptAssertion" id="68a7abcd-c568-45be-9f43-5245dd47f3a6" name="xsi:schemaLocation attribute exists">
    <con:configuration>
        **<scriptText>import de.interactive_instruments.etf.suim.*
Assert a = new Assert(messageExchange, context, log, Assert.INSPIRE_DS_NS);
a.exists("/*:WFS_Capabilities/@*:schemaLocation", "TR.missingSchemaLocation");</scriptText>**
    </con:configuration>
</con:assertion>
<con:assertion type="Simple Schema Validator" name="Response validates against schema" id="f19a8f8e-9649-4910-9073-e15c0ae2a7f9">
    <con:configuration>
        <pathToXSD>xsi:schemaLocation</pathToXSD>
    </con:configuration>
</con:assertion>
<con:credentials>
    <con:selectedAuthProfile>Basic</con:selectedAuthProfile>
    <con:addedBasicAuthenticationTypes>Basic</con:addedBasicAuthenticationTypes>
    <con:authType>Global HTTP Settings</con:authType>
</con:credentials>
<con:jmsConfig JMSDeliveryMode="PERSISTENT"/>
<con:jmsPropertyConfig/>
<con:parameters>
    <con:parameter>
        <con:name>REQUEST</con:name>
        <con:value>GetCapabilities</con:value>
        <con:style>QUERY</con:style>
        <con:type
            xmlns:xs="http://www.w3.org/2001/XMLSchema">xs:string
        </con:type>
        <con:default>GetCapabilities</con:default>
        <con:description xsi:nil="true"/>
    </con:parameter>
    <con:parameter>
        <con:name>SERVICE</con:name>
        <con:value>${#TestSuite#service}</con:value>
        <con:style>QUERY</con:style>
    </con:parameter>
    <con:parameter>
        <con:name>ACCEPTVERSIONS</con:name>
        <con:value>${#TestSuite#version}</con:value>
        <con:style>QUERY</con:style>
    </con:parameter>
    <con:parameter>
        <con:name>VERSION</con:name>
        <con:value>${#TestSuite#version}</con:value>
        <con:style>QUERY</con:style>
    </con:parameter>
</con:parameters>undefined</con:config>
iuriemaxim commented 3 years ago

@fhoubie For point 2 you may consult the OGC 06-121r3:2009 that is mentioned in the WFS standard on how to encode the GetCapabilities response.

image

All provided examples in OGC 06-121r3:2009 are with schemaLocation on the root, including for a GetCapabilities response, as can be seen below:

image

fhoubie commented 3 years ago

@iuriemaxim It is just an example, one way of doing it. Any way of specifying schemaLocation, as far as it follows the XML Schema rules is correct. No specification can constrain the XML Specification. Thus, the test can't force the schemaLocation to be at the root of the XML document as the XML schema specification allows it. And there is indeed no ATS that mandates schemaLocation to be on the root, only an ATS that states the XML shall be compliant to the schemas.

https://www.w3.org/TR/xmlschema-1/#schema-loc

  1. xsi:schemaLocation and xsi:noNamespaceSchemaLocation [attributes] can occur on any element. However, it is an error if such an attribute occurs after the first appearance of an element or attribute information item within an element information item initially ·validated· whose [namespace name] it addresses. According to the rules of Layer 1: Summary of the Schema-validity Assessment Core (§4.1), the corresponding schema may be lazily assembled, but is otherwise stable throughout ·assessment·. Although schema location attributes can occur on any element, and can be processed incrementally as discovered, their effect is essentially global to the ·assessment·. Definitions and declarations remain in effect beyond the scope of the element on which the binding is declared.
iuriemaxim commented 3 years ago

@fhoubie

If so, why for point 1 OGC has this ATS?

image

That's why points 1 and 2 has something in common, For point 1 there is an ATS based on an example.

I think that the only solution is to clarify with OGC the context of the examples in the OGC standards, if they should be just treated as examples of one way of doing it, for both for point 1. GMLs and for point 2. GetCapabilities XML response.

For point 1 you may request OGC to withdraw the test ATS A.3.1 as the it is based on an example, not on a clear requirement.

fhoubie commented 3 years ago

@iuriemaxim

  1. in OGC specifications, an example is never a requirement, only requirements are requirements. The ATS shall allow verification of the requirement. That is indeed a first rule to apply. Only requirements are normative, example are always informative.
  2. For point , a GML document needs a schema from the root to validate, because the schema gives the structure of the GML document, that contains instances of the Feature type. So indeed, for a GML document, the schemaLocation has to be on the root otherwise it is not possible to understand the structure of the Feature.
  3. a Capabilities document is not a GML document, so this rule does not apply, XML Schema rules apply.
iuriemaxim commented 3 years ago

@fhoubie But there is no requirement in the GML standard that specifies that schemaLocation should be in the rooot. There is only an example and an ATS.

For the GetCapabilities XML document also there are no specific requiremnts, just an example.

I see your point regarding examples in the OGC specifications, but the first point contradicts it, as there is a OGC ATS test mentioned in the standard, based on an example and not based on a clear requirement (for GML).

Regarding the fact that GML needs a schema in the root to validate, if I remember well, a GML is considered valid by several validators even without the xsi:schemaLocation element as far as the schemas are accessible at their URLs. But once again, here you are explaining that the examples in the OGC specifications should be seen just as examples and if no requirements exist clearly, they do not exist.

As I mentioned, I think that only OGC can clarify this. If a standard is not clear, then a new version is expected.

lbarania commented 3 years ago

Dear @iuriemaxim,

I think that you can agree that a concept not defined by a more specific document should first, by default, be consulted by normative references of said document. Only after such chain of authority has been followed and no answer found, common sense should be applied based off other applicable information (like examples). However, inferring requirements from examples is an overinterpretation of a specification.

The XML Schema (both the 2001 and the 2004 versions) clearly indicate the valid placement of the xsi:schemaLocation and xsi:noSchemaLocation attributes hinting the location XML Schema instances for the validator. Corollarily, it is true that a validator might outright ignore the hints if it believes it has the authoritative version of the XML Schemas describing given elements (as inferred by the QName).

In the case of GML ATS, the ATS.3.1 and ATS.3.2 are actually redundant reiterations of ATS.3.4 that is based off the XML Schema spec. One cannot validate against an XML Schema a document whose root element is defined in an XML Schema that's unknown to the validator. No issue here.

As for the Capabilities XML validation: To our knowledge, there's no xsi:schemaLocation placement requirement in neither INSPIRE TG DS 1.0, nor OGC WFS 2.0, nor OGC OWS 1.1. The only requirement is that the XML shall validate against appropriate XML Schemas. Hence, usual XML Schema spec rules apply, which have been already quoted to you. In this case, the INSPIRE validator places stricter requirements on the implementation than the specifications it is validating against.

iuriemaxim commented 3 years ago

@lbarania Following the logic described, no ATS and test should exist if no requirement exist. Can you please indicate the document and requirement that specify that in a GML the xsi:schemaLocation should be in the root? I am not expecting the ATS based on the example to be provided as this contradicts the logic behind.

lbarania commented 3 years ago

@lbarania Following the logic described, no ATS and test should exist if no requirement exist. Can you please indicate the document and requirement that specify that in a GML the xsi:schemaLocation should be in the root? I am not expecting the ATS based on the example to be provided as this contradicts the logic behind.

@iuriemaxim This issue relates to the validator putting overly strict requirements on the contents of the capabilities xml returned from an implementation being validated. Please provide justification for the xsi:schemaLocation being required to be placed at the root element. We are aware of none such requirement being imposed by the specs the validator is supposed to be following.

iuriemaxim commented 3 years ago

My question was in relation with the xsi:schemaLocation requirement in GML, not within the GetCapabilities.

fhoubie commented 3 years ago

@iuriemaxim GML specification was written before the existence of the OGC Modular Specification that standardize the concept of Conformance Classes and Requirements. So, the GML specification does not have the concept of requirements, but the ATS of GML shall allow validation.

The issue here is not at all related to OGC specification but XML understanding.

lbarania commented 3 years ago

Dear @iuriemaxim

Since you insist

My question was in relation with the xsi:schemaLocation requirement in GML, not within the GetCapabilities.

Then to answer your question:

Can you please indicate the document and requirement that specify that in a GML the xsi:schemaLocation should be in the root?

The document is the specification of ISO 19136:2007 which in turn is the same as OGC 07-036. The specific requirement is described as part of Annex A that normatively describes test cases that GML application schemas, GML profiles and GML documents need to pass to be deemed GML conformant. The specific requirement you ask about is A.3.1. You already quoted that in your previous comment at https://github.com/inspire-eu-validation/community/issues/398#issuecomment-707020987

Now. Can you please return the favor and reply to my inquiry as to the justification for the validator requiring the presence of the xsi:schemaLocation attribute in the root element of the capabilities document, for which this issue was filed?

(edit) Just to be perfectly clear. Given all the evidence in the specification documents, we believe the validator imposes too strict of a requirement. If you point us to a normative reference describing such a requirement, we will gladly comply. The arguments put forth so far don't indicate this to be the case. If the validator strictness stems from a technical limitation (unlikely), we will not so gladly comply. If you don't know how to address the issue (unlikely) we will grumpily comply.

iuriemaxim commented 3 years ago

The OGC 06-121r3:2009, WFS and the GML standard are not made according to OGC Modular Specification. These standards are using examples. In the GML standard the examples are used as a base for ATS. Then why in the OGC 06-121r3:2009 standard the examples should be treated different?

I think that the validator requires the presence of the xsi:schemaLocation attribute based on the interpretation of the OGC 06-121r3:2009, section "7.4.10 Service metadata XML example", at page 40.

iuriemaxim commented 3 years ago

I looked at this GetCapabilities document mentioned in post 1 http://sdidemo.intergraph.pl/WFS_INSPIRE/service.svc/get?ACCEPTVERSIONS=2.0.0&REQUEST=GetCapabilities&SERVICE=WFS&VERSION=2.0.0

One important problem is that the GetCapabilities document is including in the root xmlns="http://www.opengis.net/wfs" Opening the http://www.opengis.net/wfs link in the browser it can be seen that this is the schema for WFS 1.1 http://schemas.opengis.net/wfs/1.1.0/wfs.xsd But INSPIRE requires WFS 2.0

Once the WFS 2.0 schema will be referenced the xsi:SchemaLocation will be necessary to map: http://www.opengis.net/wfs/2.0 http://schemas.opengis.net/wfs/2.0/wfs.xsd

xsi:SchemaLocation is also needed in order to map the inspire_dls schema used for Extended Capabilities. http://inspire.ec.europa.eu/schemas/inspire_dls/1.0 http://inspire.ec.europa.eu/schemas/inspire_dls/1.0/inspire_dls.xsd

You may consider to look at the https://inspire.meteoromania.ro/WIGOS/WFS?service=WFS&request=GetCapabilities as an example

lbarania commented 3 years ago

In the GML standard, it would be a blunder to provide examples that don't pass the appropriate ATSes. Saying that ATS are based off examples is an overinterpretation. The GML standard very clearly states in its section 2 that conformant applications, profiles and documents shall follow rules specified in relevant clauses and ATSes. Same thing applies to the OWS Common document, which in its section 2 titled conformance, has this to say:

Conformance with this specification shall be checked using all the relevant abstract tests specified in the Abstract Test Suite provided in Annex A of this specification. More specifically, all the relevant abstract tests in Annex A shall be included or referenced in the Abstract Test Suite in each separate specification that normatively references this specification.

No mention of following examples.

lbarania commented 3 years ago

@iuriemaxim That argument is moot. Even though the default namespace is set to http://www.opengis.net/wfs, the root element is using the prefix wfs which is mapped correctly to http://www.opengis.net/wfs/2.0

This implementation passes the OGC CITE certification which correctly requires the capabilities XML document to validate against the WFS 2.0 schema.

iuriemaxim commented 3 years ago

And this http://inspire.ec.europa.eu/schemas/inspire_dls/1.0 ?

lbarania commented 3 years ago

Correctly referenced where is needed - the INSPIRE ExtendedCapabilities.

(edit) <ExtendedCapabilities xmlns="http://inspire.ec.europa.eu/schemas/inspire_dls/1.0" xsi:schemaLocation="http://inspire.ec.europa.eu/schemas/inspire_dls/1.0 http://sdidemo.intergraph.pl/WFS_INSPIRE/Schemas/inspire/inspire_dls/1.0/inspire_dls.xsd">

iuriemaxim commented 3 years ago

Good, so I expect that this can be marked as discussion by @carlospzurita and the validator should not be changed in order not trigger an error if the xsi:schemaLocation is missing (in the root).

Once the GetCapababilities will look correct it will be seen that it is necessary to insert also the http://www.opengis.net/wfs/2.0 http://schemas.opengis.net/wfs/2.0/wfs.xsd as the first element in the xsi:schemaLocation.

Why is this necessary, I have no ideea, but as the first prefix is wfs, it should be present in the xsi:schemaLocation for a proper validation in a XML validator. I do not think that the INSPIRE validator is looking at the order of the elements in the xsi:schemaLocation as it is not also looking at the header at all ... unfortunately. There were some issues reported on this subject.

lbarania commented 3 years ago

@iuriemaxim While I appreciate you saying that you don't know why it's necessary, I can't agree with your conclusions.

It is not necessary to have the xsi:schemaLocation present at all to be able to validate an XML file using XML Schema. An XML Schema validator can be imbued with upfront knowledge of XML Schema documents to validate the XML against.

Please advise whether the requirement for INSPIRE compliance is to have the capabilities XML reference the XML Schema documents in a specific way, or to pass XML Schema validation.

iuriemaxim commented 3 years ago

I think that @lbarania, your questioning why it is necessary the xsi:schemaLocation, as for you the extendedCapabilities argument is fine, while I think that @fhoubie is asking why it is necessary to be present in the root, as he is using the xsi:schemaLocation for the INSPIRE extended capabilities as such: image

At least here, the things are clear, as the TG for Download services has for example these requirements

image

while in the service mentioned at point 1 it looks like image

and correct would be image

The service has quite many issues that need to be fixed before raising issues as this one.

lbarania commented 3 years ago

Can you please elaborate on why do you think the XML is wrong, @iuriemaxim? Or are you referring to other problems?

(edit) Oh, I see you're referring to other problems. The email notification cut short. Still. The XML is valid. The prefix is not mandatory.

Łukasz


From: iuriemaxim notifications@github.com Sent: Monday, October 12, 2020 11:28:11 PM To: inspire-eu-validation/community community@noreply.github.com Cc: BARANIAK Lukasz lukasz.baraniak@hexagon.com; Mention mention@noreply.github.com Subject: Re: [inspire-eu-validation/community] Validator complains schemaLocation is not defined at root element (#398)

This email is not from Hexagon’s Office 365 instance. Please be careful while clicking links, opening attachments, or replying to this email.

I think that @lbaraniahttps://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Flbarania&data=02%7C01%7C%7C3c606b1a70664977c33308d86ef5b913%7C1b16ab3eb8f64fe39f3e2db7fe549f6a%7C0%7C0%7C637381348935196655&sdata=cooDrw%2BVEXgEbIkrtQVKbeYFyvonI4%2FpYP5eMl6urOw%3D&reserved=0, your questioning why it is necessary the xsi:schemaLocation, as for you the extendedCapabilities argument is fine, while I think that @fhoubiehttps://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ffhoubie&data=02%7C01%7C%7C3c606b1a70664977c33308d86ef5b913%7C1b16ab3eb8f64fe39f3e2db7fe549f6a%7C0%7C0%7C637381348935196655&sdata=9w0KvodGNmgmXkDWr%2FTwdkKlHxHrGPKFW2NPnx6pkXs%3D&reserved=0 is asking why it is necessary to be present in the root, as he is using the xsi:schemaLocation for the INSPIRE extended capabilities as such: [image]https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fuser-images.githubusercontent.com%2F20813598%2F95791518-30784100-0cea-11eb-858c-3632b6f58b34.png&data=02%7C01%7C%7C3c606b1a70664977c33308d86ef5b913%7C1b16ab3eb8f64fe39f3e2db7fe549f6a%7C0%7C0%7C637381348935206651&sdata=4pJNaUB4bHOq75tLfTJweJgMhJKkKy%2BzoUF2QHXLAyI%3D&reserved=0 while an example that passes the tests would be [image]https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fuser-images.githubusercontent.com%2F20813598%2F95791697-949b0500-0cea-11eb-82da-5f38e0da0c91.png&data=02%7C01%7C%7C3c606b1a70664977c33308d86ef5b913%7C1b16ab3eb8f64fe39f3e2db7fe549f6a%7C0%7C0%7C637381348935206651&sdata=ppOQ9d2iE1fFF0ULfTFh5TitlmPEJoe9h1cE16Db9eg%3D&reserved=0

I just think that the WFS mentioned in post 1 is so wrong that first should be fixed its major issues.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Finspire-eu-validation%2Fcommunity%2Fissues%2F398%23issuecomment-707354418&data=02%7C01%7C%7C3c606b1a70664977c33308d86ef5b913%7C1b16ab3eb8f64fe39f3e2db7fe549f6a%7C0%7C0%7C637381348935216647&sdata=88B0Lxll0e6vUKzOj21960NAkw8rUMshg7sL9SvQl98%3D&reserved=0, or unsubscribehttps://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FANICZMQ6S4U5F3RPQZXJYMTSKNYGXANCNFSM4R4KPFVQ&data=02%7C01%7C%7C3c606b1a70664977c33308d86ef5b913%7C1b16ab3eb8f64fe39f3e2db7fe549f6a%7C0%7C0%7C637381348935216647&sdata=P4By18ZU4d4IyriViyOmyrDKyifEzS%2BFi8zQSS27taE%3D&reserved=0.

iuriemaxim commented 3 years ago

Can you please elaborate on why do you think the XML is wrong, @iuriemaxim? Or are you referring to other problems? Łukasz

I did not said that the GetCapabilities XML is wrong as I see that your focus is just to be compliant with XML standard and to validate against the schema.

I said that the service (WFS) has quite many issues that need to be fixed I already mentioned some: it is advertised as WFS 1.1 instead of 2.0 It does not fulfill requirements 58 and 59 from the Download TG, The WFS test shows quite many errors, many of them being triggered by OGC, not by the INSPIRE validator image

lbarania commented 3 years ago

Am I correct to understand that you don't find the current issue as valid, because of other issues the WFS implementation might have, @iuriemaxim?

iuriemaxim commented 3 years ago

No, I just said that when a service has so many issues is better to try to solve some of them before raising such issues. This particular issue stil exist as the xsi:schemaLocation is not found in the root. I just see that there are so many data providers that are just asking the validator tests to be relaxed because their services are not passing the validator. It is easy to remove tests and not to ensure interoperability.

For this particular issue, first the requirements 58 and 59 from the Download TG should be fulfilled.

lbarania commented 3 years ago

Both requirements you mentioned are fulfilled, @iuriemaxim. Please advise why you think they're not.

iuriemaxim commented 3 years ago

It does not look like this, as specified in the requirements image and I already know what you will say. Lets wait for the JRC to advise on how to interpret the TG.

lbarania commented 3 years ago

I would say that you have a problem with understanding specs. The xml doesn't look like "this", but still it means the same. Please take a look at https://www.w3.org/TR/xml-names/ I would also say that you steer the discussion in every direction other than addressing a very specific issue raised. It would be much better if you closed it as "won't do".

iuriemaxim commented 3 years ago

I agree that it means the same, but I do not agree that it comply with the TG requirements (inspire_common prefix is missing) and I told you that I knew what you will say. If in INSPIRE there is a requirement to use a prefix, lets say "ps", than it is not allowed to use "ps1" instead of "ps", even if both ps and ps1 would indicate the same target schema. But these are the rules even if the XML is valid and has the same meaning. However, I am not involved in bug fixing, nor I am working for the validator, but I was involved long time ago in drafting ATSs.

lbarania commented 3 years ago

That's understandable, @iuriemaxim. Since you don't work on bug fixing nor the validator, I guess we'll need to wait for somebody to authoritatively assess the issue.

iuriemaxim commented 3 years ago

The issue will disappear once the requirements from the Download TG will be met.

lbarania commented 3 years ago

Please advise the exact requirement of the TG mandating the existence of schemaLocation on the root element, @iuriemaxim.

iuriemaxim commented 3 years ago

I said that the issue will disappear once the requirements from the Download TG will be met, Namely requirements 58 and 59, as the "inspire_common" prefix should be present in the GetCapabilities response. Here JRC or other experts can clarify if the requirement is not clear enough. But I remember long discussion on forums and on TWGs regarding the prefixes, and I provided an example with ps vs ps1, where ps stands for Protected Site. One discussion that would not be held if it was possible to use both ps and ps1 for the same target schema is this one, held long time ago https://inspire.ec.europa.eu/forum/discussion/view/88206/it-seems-that-there-is-no-fully-compliant-solution-to-serve-multiple-harmonised-datasets-trough-wfs-20

Regarding the schemaLocation presence at the root, I mentioned that we have a different interpretation of the OGC 06-121r3:2009 standard, because I see examples as part of the standard, similar as for GML and WFS standard, where ATS are based on examples. Maybe I am wrong, but here OGC can clarify or we can conclude that the standard is not clear enough. Without examples is very difficult to implement a standard.

fhoubie commented 3 years ago

@iuriemaxim Examples are informative, not normative. XML Schema is normative. ATS cannot be based on examples. Examples are by definition showing an example of what it could be, not the example.

If in the example the schemaLocation attribute is the 4th in the root element, shall the test check that the implementation follows that. I have some doubts. It is not about clarity, it is about applying Applicable and Reference documents.

fhoubie commented 3 years ago

Hi @michellutz, @MarcoMinghini, could someone from JRC have a look here ? Thanks

iuriemaxim commented 3 years ago

@iuriemaxim Examples are informative, not normative. XML Schema is normative. ATS cannot be based on examples. Examples are by definition showing an example of what it could be, not the example.

If in the example the schemaLocation attribute is the 4th in the root element, shall the test check that the implementation follows that. I have some doubts. It is not about clarity, it is about applying Applicable and Reference documents.

Of course that examples should be seen as examples. I asked before on what requirement is based the ATS 3.1. in the GML standard The answer provided, I interpreted as being that is based on ATS 3.1.

ingosimonis commented 3 years ago

Please allow me to weigh in here. The discussion seems to go in circles and from time to time departs to other directions. I am addressing the initial issue, i.e. the question if the xsi:schemaLocation attribute is required to be located in the root element or not for specific responses to a WFS instance. To my understanding, @fhoubie is correct in his remarks. It is important to differentiate between requirements that are specific to GML, and requirements that are specific to a service type (here WFS/GetCapabilities).

OGC standards use examples to illustrate how a specific aspect can be implemented in conformance with the standard. Other implementations may exist in addition to what is shown in an example. Examples shall always be in conformance with the standard.

@iuriemaxim if you think that the standard has an issue that needs to be fixed, please feel free to file a request at the OGC Standards Tracker.

iuriemaxim commented 3 years ago

@ingosimonis Can you please explain why in the GML standard there is the ATS 3.1 and why it was considered critical to have the schema declarations in the root (although there is no requirement for this, but this was probably considered necessary as a first step in the test). I think that understanding the reason of a certain requirements/ATS imposed by a standard could be useful for this topic.

carlospzurita commented 3 years ago

Dear all, seeing that this is a complex topic with no clear solution, we are opting for now to relax the test and accomodate

There is no requirement supporting this check, and this situation doesn't seem to be conflicting with any specification on XML or WFS, the getCapabilities document seems to be correct; although the example document in the specification may lead to confusion.

As such, we are proposing the removal of the initial check. To validate against the schema, a fixed version of the schema will be set in the ETS. If schemaLocation attribute is present, the document would be validated against the declared schema instead.

iuriemaxim commented 3 years ago

@carlospzurita

If schemaLocation attribute is present, the document would be validated against the declared schema instead.

then data providers could declare any other schemas, including their own schema for the extended capabilities. It was another issue exactly on this topic.

  1. Are then tests in place or will be developed to check that the expected elements are present in the extended capabilities and that they will be encoded according to requirements that exist in the Download TG?

I still expect the service above not to pass the extended capabilities tests. I still think that the solution to comply with Requirements 58 and 59 from the Download TG is to have the schemaLocation declared in the root.

  1. Can you please clarify if the intention is to apply this only for Download services or for any other XML tests?

  2. Considering that data providers will be able to create their own extended capabilities schemas, can you please clarify if supplementary elements in the extended capabilities could be provided? Does TG allows this.

Please ensure that the damage will not be bigger as it already happened with other tests.

I think that the solution would be to test if it is possible to generate a GetCapabilities document that will comply with the requirements set in the Download TG, including 58 and 59, without having the schemaLocation declared in the root. If this is possible, than the schema location attribute in the root can be removed. But this does not imply that if schemaLocation attribute is present, the document should be validated against the declared schema instead, without any further checks.

iuriemaxim commented 3 years ago

@carlospzurita If there is an intention to implement this change, please do not do it unless @fhoubie will be able to fix his service (see #414, #411 and #384 related to custom extended capabilities) and him or another user can prove that it is possible to create a WFS with a GetCapabilities document where the INSPIRE schemas are not declared in the root while the GetCapabilities document will be still correct according to the Download Services TG, especially in relation with Extended Capabilities.