Pre-processing in BaseX testdriver

[jonherrmann]

ETF Improvement Proposal (EIP)

Background and Motivation:

In some cases it may be necessary to prepare the input data either by transforming the input or by ensuring that the data meets certain requirements.

A concrete use case is the sorting out of features with invalid geometries.

Proposed change

To achieve the greatest possible flexibility, it should be possible to implement the preprocessing step in XQuery. The test driver should execute the preprocessing step if an XQuery file 'prepare-chunk.xq' is found in the ETS project directory. It will be executed on every chunk before data is added to the database. The test driver provides XQ-interfaces to mark invalid test data in a file.

The docs.etf-validator.org developer documentation should point to the interface documentation in the test driver repository.

Alternatives

none

Funding

yes

Additional information

Technical details:

mark invalid test data in a file means that the specific node is not inserted into the database but moved to a separate, isolated database. This allows to remove Features with invalid geometries from further tests but with the option to access data from the feature e.g. collecting and outputting their gml IDs.

[cportele]

Approved in SG meeting 23

[jonherrmann]

Implemented in Version 2.1.0