Provide better guidance / policy on ATS identifiers

[bradh]

Background: I reported an issue at https://github.com/opengeospatial/ogcapi-common/issues/204, which was reasonably redirected to the NA.

OGC 10-103 provides an example of a conformance (ATS) test as: http://www.opengis.net/spec/OMXML/2.0/conf/measurement/result-measure

(also, the link to the register in Sect 5 of OGC 10-103 is broken).

The conf part is undesirable, because it makes it hard to tell the difference between an ATS clause and a conformance class.

Also, the guidance is insufficient, because not all ATS do it that way. For example, see: https://github.com/opengeospatial/geopackage/blob/master/spec/core/annexes/ats.adoc

Request the NA update the policy / guidance material.

[ghobona]

OGC 10-103 derives its concepts from the OGC's Specification Model - A Standard for Modular specifications (08-131r3) (ModSpec) policy.

Conformance classes are meant to be specified in an Abstract Test Suite (ATS) and may be referenced in an Executable Test Suite (ETS).

The ModSpec policy recognises a Conformance Class but does not recognise an ATS clause. One could therefore consider an ATS clause as an artefact of the document, whereas a Conformance Class is a part of the specification model. So whereas you would use a URI e.g. http://www.opengis.net/spec/OMXML/2.0/conf/measurement/result-measure to reference a Conformance Class, you would probably use the Document Section number e.g. Section A.2 to reference an ATS clause.

[bradh]

It is sometimes useful to be able to reference a particular ATS test case, even if there are several within a section. One example of this use is in formal traceability.

[cportele]

@bradh - I am not sure I understand your problem. An abstract test suite consists of one or more conformance classes and each of the conformance classes will have a URI following the template http://www.opengis.net/spec/{standard}/{m.n}/conf/{conformance-class}, e.g. http://www.opengis.net/spec/OMXML/2.0/conf/measurement. Each test case is a sub-resource of the conformance class in which it is defined and uses the URI template http://www.opengis.net/spec/{standard}/{m.n}/conf/{conformance-class}/{test-case}, e.g. http://www.opengis.net/spec/OMXML/2.0/conf/measurement/result-measure.

The path element conf to which the conformance classes are sub-resources could also have been ats, but now it is conf. It also makes sense, because we use req not spec for the path element under which the requirements classes are sub-resources.

Note that the ModSpec (08-131r3) also has the intermediary resources "requirements/conformance modules", but 10-103 makes them optional steps in the URI path and does not use them in the examples. In fact, I am not aware of any OGC standard that has made use of requirements/conformance modules in their URIs (same in ISO/TC 211), so I guess we could update our documents and remove the modules concept and consider the grouping into modules/sections a documentation detail (which may change and often does change during the drafting process without any effect on implementations). Note that the current revision of ISO 19105 has already removed them from the specification model.

[bradh]

@cportele The problem is that there is no consistency in the formatting or ATS relationships. The naming authority should fix that by providing guidance / policy.

I'm also not sure I see where the test case is M:1 with the conformance class. Where is that specified? I can see the situation where a test case could relate to more than one conformance class.

I also don't see where the URI template is required. If its there, it isn't obvious enough, nor is it being applied.

For concrete examples of where OGC specs didn't do it the way you describe:

• https://github.com/opengeospatial/ogcapi-common/blob/master/collections/abstract_tests/collections/ATS_rc-op.adoc
• https://github.com/opengeospatial/geopackage/blob/master/spec/core/annexes/ats.adoc#base
• https://github.com/opengeospatial/geotiff/blob/master/GeoTIFF_Standard/Detailed%20Test%20Suite/abstract_tests/Units_GeoKeys/TEST_UnitsGeoKey.adoc#unitsgeokey-test

I'd like a consistent, meaningful, unambiguously parseable identifier for every test case in every OGC standard.

[cportele]

@bradh - I see what you mean. Common Part 2 is just incorrect in its use of "ats", but this is an early draft and I assume this will be fixed in the process. But the GeoPackage and the GeoTIFF specs are not consistent with the existing policy. (And the URIs also do not resolve.)

I'm also not sure I see where the test case is M:1 with the conformance class. Where is that specified?

The ModSpec (08-131r3). See, for example, Figure C.1.

[bradh]

I would actually like to see ats used. I think that would help to distinguish between conformance tests and the ATS that support them.

Thanks for the pointer on the ModSpec. Interesting that Figure C.1 says you can't have a ConformanceTest without having a ConformanceModule.

[ghobona]

Closing this card because the original question about ats has been answered, as well as the follow-on question about the M:1 relationship.