Test Run Templates

[jonherrmann]

ETF Improvement Proposal (EIP)

Background and Motivation:

This EIP supplements EIP #26 . In addition to the creation of test objects, it should also be possible to pre-select a set of ETS and parameters and save these as one element to execute all ETSs easily together. Furthermore a test object should also be presettable.

Proposed change

Add a new model item Test Run Template item that possesses the following properties:

• an EID
• a label
• a description
• a version
• the language for reports
• a parameter list to parameterize the ETS
• a set of Executable Test Suites
• a set of Test Objects (optional)

An example instance which would run all INSPIRE Metadata ETS could look like this:

<TestRunTemplate id="EID...">
  <label>Metadata (Technical Guidance 1.3)</label>
  <description>Run INSPIRE Metadata (TG 1.3) tests against the files on network drive X</description>
  <language>de-DE</language>
  <ParameterList>
    <parameter name="Files_to_test">.*</parameter>
  </ParameterList>
  <ExecutableTestSuite>
    <!-- Conformance Class 'INSPIRE Profile based on EN ISO 19115 and EN ISO 19119' -->
    <executableTestSuite ref="EID..."/>
    <!-- Conformance class 'Metadata for interoperability' -->
    <executableTestSuite ref="EID..."/>
  </ExecutableTestSuite>
  <TestObjects>
    <!-- Test Object with all test data on network drive X -->
    <testObject ref="EID..."/>
  </TestObjects>
</TestRunTemplate>

TestRunTemplates are Metadata items and can be stored like Tags as XML in the Metadata directory. They will be read and loaded like Tags during startup.

The user should be able to:

• create a Test Run Template, by selecting one or multiple ETSs and optionally a Test Object
• delete an existing Test Run Template
• clone an existing Test Run Template and modify it
• Select and execute one Test Run Template

Cloning an existing Test Run Template would create a clone with a new EID. Changing an existing instance of a test run template should not be possible. The cloning feature could be configured through a property to activate/deactivate it:

etf.testruntemplates.cloning.allow = true/false

The default would be true for organisation-internal workflows and false for public workflows.

The REST interface endpoint would be accessible via /v2/TestRunTemplates .

If the user selects Test Run Template in the user interface and no Test Object has been pre-selected in the template, the user interface should ask which Test Object should be taken.

A draft for the workflow and the web interface is described in EIP #27 .

This EIP is about the new REST interface which should be implemented before the changes in the web interface and the implementation of the complete workflow.

Alternatives

none considered

Funding

Full funding available.

Additional information

• This EIP is backwards compatible with existing ETSs
• For the sake of simplicity, only a single test object would initially be supported in a Test Run Template.
• Since this is an addition to the existing interface, the main API version number v2 does not need to be increased, and existing external clients do not need to be modified
• This EIP requires changes in the ETF core, the data storage and the web application

[jonherrmann]

Todo:

• [x] describe configuration option to deactivate local Test Run Template cloning

[jonherrmann]

Configuration description updated inline, see 'etf.testruntemplates.cloning.allow'

[jonherrmann]

Implemented in Version 2.1.0