search

jenkinsci / ecu-test-execution-plugin

This plugin enables a platform-independent test execution. It integrates ecu.test and trace.check with Jenkins and allows report generation and its upload of automated test execution.
https://plugins.jenkins.io/ecu-test-execution/
Other
5 stars 3 forks source link

ttRunPackage does not wait for configuration to be loaded #148

Open sebastianuhl-umlaut opened 1 month ago

sebastianuhl-umlaut commented 1 month ago

Describe the bug

Test execution via Jenkins is failing due to unavailable tools. The same test can be executed locally.

In a Jenkins job a test execution is started with a block like

                ttStartTool toolName: 'ecu.test', workspaceDir: 'ECU-TEST_Workspace'
                ttRunPackage testCasePath: '***.pkg', testConfig: [tbcPath: '***.tbc', tcfPath: '***.tcf']
                ttStopTool toolName: 'ecu.test'

The test exection fails as the tools configured in the test bench configuration are not available.

Partial content of ecu.test_out.log:

2024-06-03 14:20:23.476 2356 Thread-9 DEBUG:
                        REST-API now available at 127.0.0.1:5050
2024-06-03 14:20:23.620 2356 MainThread DEBUG:
                        SWK: Catalog switching UI: Global- (False)
2024-06-03 14:20:27.286 2356 WSGI_0 DEBUG:
                        Add configuration change order.
2024-06-03 14:20:27.289 2356 WSGI_0 DEBUG:
                        Add execution ***.pkg
2024-06-03 14:20:27.296 2356 Thread-11 INFO:
                        REST API: Start Package ***.pkg
2024-06-03 14:20:27.296 2356 Thread-11 DEBUG:
                        Start execution of Package ***.pkg
2024-06-03 14:20:27.425 2356 Thread-11 DEBUG:
                        Wait for Package ***.pkg to finish
2024-06-03 14:20:27.825 2356 ManagedWorker-1*PE.Run() INFO:
                        Start package '***.pkg' with test case execution with sequential analysis job execution
2024-06-03 14:20:28.051 2356 ManagedWorker-1*PE.Run() INFO:
                        Test case "***" started
2024-06-03 14:20:28.109 2356 ManagedWorker-1*PE.Run() DEBUG:
                        TestEngine: Start ports...
2024-06-03 14:20:28.109 2356 ManagedWorker-1*PE.Run() WARNING:
                        Port missing in the test bench configuration: DIAG-CAN-FD01
2024-06-03 14:20:28.111 2356 ManagedWorker-1*PE.Run() WARNING:
                        Port missing in the test bench configuration: DIAG-DOIP-SOCKET01
2024-06-03 14:20:28.603 2356 ManagedWorker-1*PE.Run() INFO:
                        System time mode activated
2024-06-03 14:20:28.624 2356 ManagedWorker-1*PE.Run() ERROR:
                        Test step .1(Parse_Input_Check).1(Parse_Input).1(Calculation): "An error occurred when calling "parse_parameters(***)": Runtime error in function variable, line 75: TypeError, object of type 'NoneType' has no len()."
                        Package "ECU-TEST_Workspace\Packages\Parse_Input.pkg"#1
2024-06-03 14:20:28.791 2356 ManagedWorker-1*PE.Run() DEBUG_EXC_PRINT:
                        tts.testExecution.engine.TestEngine.ExecuteTestStep: UserCodeException: An error occurred when calling "parse_parameters(***)": Runtime error in function variable, line 75: TypeError, object of type 'NoneType' has no len().
                        Package "ECU-TEST_Workspace\Packages\Parse_Input.pkg"#1
2024-06-03 14:20:28.892 2356 ManagedWorker-1*PE.Run() WARNING:
                        Test case "***" aborted
2024-06-03 14:20:28.897 2356 ManagedWorker-1*PE.Run() DEBUG:
                        TestEngine: Stop ports...
2024-06-03 14:20:28.899 2356 ManagedWorker-1*PE.Run() INFO:
                        Test case "***" completed
2024-06-03 14:20:29.269 2356 Thread-11 DEBUG:
                        Extract results for Package ***.pkg
2024-06-03 14:20:29.303 2356 Thread-11 INFO:
                        REST API: Package ***.pkg successfully completed
2024-06-03 14:20:29.303 2356 Thread-11 INFO:
                        REST-API: Start the configuration
2024-06-03 14:20:29.304 2356 Thread-11 DEBUG:
                        ActiveConfigManager: Load TBC: ECU-TEST_Workspace\Configurations\***.tbc

The exception at timestamp 2024-06-03 14:20:28.624 is caused by api.ObjectApi.ConfigurationApi.OpenTestBenchConfiguration(api.CurrentTestbenchConfiguration.Filename).GetTool('***') being None despite this tool being configured in the test bench configuration.

The log seems to indicate that the REST API requests to load the configuration (2024-06-03 14:20:27.286) and start the test execution (2024-06-03 14:20:27.289) are sent without actually waiting for the configuration to load. Loading/starting of the configuration (2024-06-03 14:20:29.303) only starts after the test execution (2024-06-03 14:20:27.296-2024-06-03 14:20:29.303).

Steps to reproduce

-

Expected behavior

Test execution via Jenkins is starting the test bench configuration prior to starting the test execution.

Versions

Additional context

No response

ErikRehmTT commented 1 week ago

Hi, thanks for reporting the bug. We will have a look into it.

TT-S3-D1 commented 1 week ago

Hi @sebastianuhl-umlaut can you maybe provide a log from your pipeline execution also?