search

opengeospatial / ets-ogcapi-tiles10

Other
0 stars 0 forks source link

testlist.xml is reporting result 6 even when an IUT has passed the basic conformance class test #24

Open ghobona opened 7 months ago

ghobona commented 7 months ago

testlist.xml is reporting result="6" as shown below.

<test complete="yes" defaultResult="1"
      file="/root/te_base/work/_root_te_base_scripts_ogcapi-tiles-1.0_1.0_ctl_/ogcapi-tiles-1.0-suite/tns$Main.test"
      hasCache="no"
      local-name="Main"
      namespace-uri="http://www.opengis.net/cite/ogcapi-tiles-1.0"
      path="s0002"
      prefix="tns"
      result="6"
      type="Mandatory"/>

The expected behaviour is for result="6" to be result="1".

ghobona commented 7 months ago

It looks like if all conformance classes pass then the result is 1.

When an optional conformance class fails, even if it is not the basic (mandatory) conformance class, then the result is 6.

bpross-52n commented 1 month ago

@ghobona Thanks for reporting. I have checked this with other test suites (e.g. WCS 2.0) and I experienced the same behaviour. Thus, I think this is an issue with TEAM Engine. Edit: see https://github.com/opengeospatial/teamengine/issues/622