search

etf-validator / etf-webapp

:earth_africa: :mag: ETF is an open source testing framework for spatial data and services
https://www.etf-validator.net
European Union Public License 1.2
19 stars 19 forks source link

Test Driver stops loading OGC WFS 2 tests after Team Engine update #207

Closed jonherrmann closed 2 years ago

jonherrmann commented 5 years ago

Please note, @etf-validator/sg.

Description

The Team Engine Test Driver uses only whitelisted test suites. After the Team Engine has been updated today, the name of the test suite has changed and is no longer loaded.

Proposed bugfix

The name has changed from WFS 2.0 (OGC 09-025r2/ISO 19142) Conformance Test Suite to WFS 2.0 (ISO 19142:2010) Conformance Test Suite. See

http://cite.opengeospatial.org/teamengine/rest/suites

Perhaps it is not so smart to rely on the full name but on the "ID", wfs20 which we can take from the link: http://cite.opengeospatial.org/teamengine/rest/suites/wfs20/1.32/ .

klimeto commented 4 years ago

Not 100% sure this is linked but today I tried to run Test Run Test run on 22:35 - 26.02.2020 with test suite Conformance Class: Download Service - Direct WFS on both INSPIRE ETF production and staging environment it got stacked on step invoking Team Engine remotely:

Log from ETF prod:

Connecting to test runner and waiting for new messages...26.02.2020 21:36:31 - Preparing Test Run Test run on 22:35 - 26.02.2020 with test suite Conformance Class: Download Service - Direct WFS (initiated Wed Feb 26 21:36:31 UTC 2020)
26.02.2020 21:36:31 - Resolving Executable Test Suite dependencies
26.02.2020 21:36:31 - Preparing 2 Test Task:
26.02.2020 21:36:31 -  TestTask 1 (a96fae7c-ecae-4b8b-8580-1cbe08894508)
26.02.2020 21:36:31 -  will perform tests on Test Object 'INSPIRE PD 38,01 Ukladacia služba Lokalizácia zariadení' by using Executable Test Suite 'WFS 2.0 (OGC 09-025r2/ISO 19142) Conformance Test Suite (EID: 95a1b6fc-2b55-3d43-9502-3b8b605bda10, V: 1.32.0 )'
26.02.2020 21:36:31 -  with parameters: 
26.02.2020 21:36:31 - testRunTags = 
26.02.2020 21:36:31 - endpt.GetCapabilities.Get = 
26.02.2020 21:36:31 -  TestTask 2 (8cf41d37-ddbf-494f-8f14-69c2f1190b2e)
26.02.2020 21:36:31 -  will perform tests on Test Object 'INSPIRE PD 38,01 Ukladacia služba Lokalizácia zariadení' by using Executable Test Suite 'Conformance Class: Download Service - Direct WFS (EID: ed2d3501-d700-4ff9-b9bf-070dece8ddbd, V: 1.0.2 )'
26.02.2020 21:36:31 -  with parameters: 
26.02.2020 21:36:31 - testRunTags = 
26.02.2020 21:36:31 - endpt.GetCapabilities.Get = 
26.02.2020 21:36:31 - Test Tasks prepared and ready to be executed. Waiting for the scheduler to start.
26.02.2020 21:36:31 - Setting state to CREATED
26.02.2020 21:36:31 - Changed state from CREATED to INITIALIZING
26.02.2020 21:36:31 - Starting TestRun.47097827-226e-4a24-974a-d9cb15b94aff at 2020-02-26T21:36:33Z
26.02.2020 21:36:33 - Changed state from INITIALIZING to INITIALIZED
26.02.2020 21:36:33 - TestRunTask initialized
26.02.2020 21:36:33 - Invoking TEAM Engine remotely. This may take a while. Progress messages are not supported.
26.02.2020 21:36:33 - Timeout is set to: 20min

Log from ETF staging:

Connecting to test runner and waiting for new messages...26.02.2020 21:42:13 - Preparing Test Run Test run on 22:42 - 26.02.2020 with test suite Conformance Class: Download Service - Direct WFS (initiated Wed Feb 26 21:42:13 UTC 2020)
26.02.2020 21:42:13 - Resolving Executable Test Suite dependencies
26.02.2020 21:42:13 - Preparing 2 Test Task:
26.02.2020 21:42:13 -  TestTask 1 (0dff950a-b97e-468d-bf12-512de80aaaeb)
26.02.2020 21:42:13 -  will perform tests on Test Object 'INSPIRE PD 38,01 Ukladacia služba Lokalizácia zariadení' by using Executable Test Suite 'WFS 2.0 (OGC 09-025r2/ISO 19142) Conformance Test Suite (EID: 95a1b6fc-2b55-3d43-9502-3b8b605bda10, V: 1.32.0 )'
26.02.2020 21:42:13 -  with parameters: 
26.02.2020 21:42:13 - testRunTags = 
26.02.2020 21:42:13 - endpt.GetCapabilities.Get = 
26.02.2020 21:42:13 -  TestTask 2 (f86d0984-fdf7-4dfc-aa65-e4feeaa0da7d)
26.02.2020 21:42:13 -  will perform tests on Test Object 'INSPIRE PD 38,01 Ukladacia služba Lokalizácia zariadení' by using Executable Test Suite 'Conformance Class: Download Service - Direct WFS (EID: ed2d3501-d700-4ff9-b9bf-070dece8ddbd, V: 1.0.3 )'
26.02.2020 21:42:13 -  with parameters: 
26.02.2020 21:42:13 - testRunTags = 
26.02.2020 21:42:13 - endpt.GetCapabilities.Get = 
26.02.2020 21:42:13 - Test Tasks prepared and ready to be executed. Waiting for the scheduler to start.
26.02.2020 21:42:13 - Setting state to CREATED
26.02.2020 21:42:13 - Changed state from CREATED to INITIALIZING
26.02.2020 21:42:13 - Starting TestRun.0d32276c-08ce-4991-8ad3-433b831a95d3 at 2020-02-26T21:42:15Z
26.02.2020 21:42:15 - Changed state from INITIALIZING to INITIALIZED
26.02.2020 21:42:15 - TestRunTask initialized
26.02.2020 21:42:15 - Invoking TEAM Engine remotely. This may take a while. Progress messages are not supported.
26.02.2020 21:42:15 - Timeout is set to: 20min

Even the last message says timeout is set to 20 min

jonherrmann commented 2 years ago

Related to https://github.com/etf-validator/etf-webapp/issues/223

Fixed in Version 2.1