New test driver for web service tests

[jonherrmann]

ETF Improvement Proposal (EIP)

Background and Motivation:

Add a new test driver to support tests on test object types that represent Web services and Web APIs conforming to specifications.

Currently the following test drivers exist:

• BaseX-based tests mainly for XML-based test objects;
• SoapUI-Open-Source-based tests mainly for test objects that are web services;
• TEAM-Engine-based tests mainly for OGC CITE tests that can be invoked via HTTP.

The future of SoapUI Open Source is now unclear (rare releases with minimal changes: Feb 2019, Nov 2017; no Java 11 support). In addition, SoapUI introduces limitations on how web service tests can be specified and requires workarounds that result in test suites that are difficult to understand. An overview of the current limitations:

• Usability
  • limitations on what information can be shared in parameterised tests
  • dependencies are hard to understand for the user
• Efficiency
  • no XSD cache results in slow validation and higher network usage
  • all data in memory only, no tests with larger amounts of data
• Maintainability
  • parametrised tests require scripting
  • dependencies are difficult to express
  • test steps and assertions cannot be re-used
• Portability
  • no Java 11 support
• Functionality
  • no support for pre-defined schemas

[cportele]

Architecture diagram presented in the 21st ETF SG meeting: