MicroFocus / ADM-FT-ToolsLauncher

Other
12 stars 8 forks source link

Re-running failed tests results in wrong exit code #68

Closed metaph3r closed 2 years ago

metaph3r commented 2 years ago

Tool FTToolsLauncher_net48 v1.0.23.2025 (v1.0 beta (rev.7))

Describe the bug If a test fails when run with FTToolsLauncher it quits with exit code -1. This signals the CI/CD-Tool (TeamCity) in this case there was an error which can then in turn be used to define the following build steps. The the build itself will than finish as failed which is expected bahavior.

But if Reruns is used, a test fails and then fails again in the rerun the FTToolsLauncher will quit with exit code 0 which signals that the step was successful.

To Reproduce Steps to reproduce the behavior:

  1. Configure fttooslauncher.conf with Reruns1=1
  2. Have a UFT Test which always fails
  3. Launch FTToolsLauncher with the UFT test
  4. See exit code of FTToolsLauncher

Expected behavior If at least 1 tests fails and fails again in its rerun the exit code should be -1. Only if all tests succeed after a rerun the exit code should be 0.

Log

[11:52:49] : [Step 1/4] Run status: Job failed, total tests: 3, succeeded: 0, failures: 3, errors: 0, warnings: 0 [11:52:49] : [Step 1/4] Failed : C:\Temp\demo\UFT1502_GUI_Flight[1] [11:52:49] : [Step 1/4] Failed : C:\Temp\demo\TXT-File[2] [11:52:49] : [Step 1/4] Failed : C:\Temp\demo\UFT1502_GUI_Flight[3] [11:52:49] : [Step 1/4] ================================================ [11:52:49] : [Step 1/4] Process exited with code 0

fttoolslauncher.conf

Basic parameters

runType=FileSystem resultsFilename=results.xml resultTestNameOnly=true

File System parameters

fsTimeout=3600 Test1=C:\Temp\demo\UFT1502_GUI_Flight Test2=C:\Temp\demo\TXT-File

Rerun parameters

onCheckFailedTest=true testType=Rerun only failed tests Reruns1=1

metaph3r commented 2 years ago

CICDTT_Run_UFT_Test_37.log Testlog with rerun and exit code 0

metaph3r commented 2 years ago

CICDTT_Run_UFT_Test_19.log Testlog without rerun and exit code -1

soleverxu commented 2 years ago

Thank you for raising the issue. We are now investigating the root cause and will update later.

soleverxu commented 2 years ago

The exit code problem is fixed in v1.0 beta (rev.9). Please down the FTToolsLauncher v1.0.29.221.