search

MicroFocus / ADM-FT-ToolsLauncher

Other
12 stars 8 forks source link

FTToolsLauncher - Error counting test suite failed tests #98

Open HApolonio opened 4 months ago

HApolonio commented 4 months ago

Tool

FTToolsLauncher

Bug Description

Since version 1.0 beta (rev. 11) there was introduced a bug on the total count of failed tests inside the results XML file. The property <testsuite failures='nn' ... is always set to zero, even when there are failed tests.

Sample result for 2 tests where one of them fails: [version v1.0 beta (rev.11), and bellow]

<testsuites xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <testsuite tests="2" failures="1" errors="0" time="33.21" package="FTToolsLauncher">

[version v1.0 beta (rev.12), and above]

<testsuites xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" name="uftRunnerRoot">
  <testsuite tests="2" failures="0" errors="0" skipped="0" package="FTToolsLauncher">

⚠ Notice the property failures="0"

How To Reproduce

Steps to reproduce the behavior:

  1. Create 2 GUI tests with UFT One (with the latest version), that just wait for one second and then exit.
  2. Force one of the tests to fail, using a ReportEvent micFail.
  3. Place both tests on a .MTB file, named Test_Plan.mtb.
    [Files]
NumberOfFiles=2
File1=C:\Temp\FTToolsLauncher bug\Tests\GUITest1;1
File2=C:\Temp\FTToolsLauncher bug\Tests\GUITest2-Fail;1
  4. Create a parameters file for FTToolsLauncher, named FTToolsLauncher_net48_v1-0r12_params.txt with the following content:
    resultsFilename=.\\Results\\results_v1-0r12.xml
Test1=.\\Test_Plan.mtb
runType=FileSystem
fsUftRunMode=Fast
onCheckFailedTest=false
testType=Rerun only failed tests
Reruns1=0 
fsTimeout=1800
  5. Run the tests using FTFoolsLauncher v1.0 beta (rev.12), and using the latest version of UFT One.
    .\FTToolsLauncher_net48_v1-0r12.exe -paramfile FTToolsLauncher_net48_v1-0r12_params.txt
  6. After the execution check the contents of the results file .\Results\results_v1-0r12.xml.

Expected behavior

The XML tag testsuite should have the values:

<testsuite tests="2" failures="1" errors="0" ...

Giving the correct sum of failed tests on the suite, but it reports 0 (zero) failures.

OS and testing tools

[EDIT] ℹ Just to make it clear that this bug is present on all the versions, since rev.12 until the latest one 24.2 ℹ

HApolonio commented 4 months ago

ℹ Just to make it clear: I checked that this bug is present on all the versions after rev.11. It was introduced in rev.12 and is present until the latest 24.2 ℹ

dorin7bogdan commented 4 months ago

Hi @HApolonio , We are going to open an internal bug for this issue. If this bug it's a blocker for you and need an urgent hotfix, please contact our Support at: https://www.microfocus.com/en-us/contact-support/stackb and request them to open a ticket for UFT One project, and to assign it to the team responsible of ADM-FT-ToolsLauncher or ADM-TFS-Extension.

HApolonio commented 3 months ago

Hi @dorin7bogdan, Is there any prediction for when this bug could be resolved? (1 month, 6 months, 1 year, ...)

dorin7bogdan commented 3 months ago

Hi @HApolonio, Currently the FTToolsLauncher was not scheduled for next release. However, if you need a private hotfix quicly, please contact our Support at: https://www.microfocus.com/en-us/contact-support/stackb and request them to open a ticket for UFT One project, and to assign it to the team responsible of ADM-FT-ToolsLauncher or ADM-TFS-Extension. Thank you for understanding.