Configuration of conformance class to represent the basic conformance class in the TestNG test and CTL test.

[keshavnangare]

For TestNG test:

To represent the basic conformance class in TestNG test following configuration required into config.xml of the corresponding test:

Add following elements in the suite-element

<conformanceClasses>
    <conformanceClass>GML Documents</conformanceClass>
</conformanceClasses>

As per the above configuration, it will add the isBasic attribute in the earl report.

For CTL test:

Add conformance class configuration attribute in the test element:

For example: <ctl:test name="getcapabilities:main" isConformanceClass="true" isBasic="false">

"isBasic": Represent the basic conformance class. Its value is 'true' iff that conformance class is basic otherwise 'false'.

[bermud]

The solution should allow to have multiple basic conformance classes.

[rjmartell]

I suggest adding an "optionality" parameter to each conformance class (test element) in the TestNG config file to indicate its optionality with respect to a conformity assessment. For example:

<suite name="ets-abc10">
  <test name="Conformance Class Alpha">
    <parameter name="optionality" value="mandatory">
    All constituent tests are run to verify mandatory conformance criteria.
    </parameter>
    <packages>
      <package name="org.example.ets.abc10.alpha.*" />
    </packages>
  </test>
  <test name="Conformance Class Beta">
    <parameter name="optionality" value="recommended">
    All constituent tests are run. Failing tests may be flagged as warnings.
    </parameter>
    <packages>
      <package name="org.example.ets.abc10.beta" />
    </packages>
  </test>
</suite>

All conformance classes marked as "mandatory" constitute the minimal conformance criteria. Some specifications also stipulate recommendations that should be met; perhaps if any of these tests fail they could be flagged in the report as a warning. By default, a conformance class is optional.

[rjmartell]

The optionality parameter for a conformance class shows up in the EARL report as a property of an earl:TestRequirement resource:

<earl:TestRequirement>
    <dct:title>Conformance Class Alpha</dct:title>
    <cite:optionality>mandatory<cite:optionality>
    ...
</earl:TestRequirement>

[rjmartell]

A "Basic" test in ISO 19105 is just an "initial capability test" that acts like a precondition. It doesn't necessarily mean that the test is traced to a mandatory requirement. The "optionality" property would have one of the following values:

• mandatory: the IUT is deemed to be non-conformant if any of these tests fail.
• recommended: the IUT passes a conformity assessment, but perhaps with a warning
• optional (default): No impact on overall conformity assessment

In general, at least one conformance class (CC) is marked as "mandatory". But some OGC specs don't do this and just stipulate that one or more CCs in some set must be satisfied. Perhaps a different value can address this possibility. e.g.

• conditional-oneOrMore: one or more of the CCs marked in this manner must pass

[bermud]

@keshav-nangare, have you reviewed these comments from Richard?