--runemptysuite should work with --rerunfailed when there are no failed tests

[efrenette11]

When we want to re-executed tests using pabot from a previous run, if there's no failure, the execution stopped.

I do have a jenkins script that will allow 3 retry for the test to passed. If all the tests passed at the first run, my script will stop.

Looks like to be the same issue that we had on robotframework and the fix it in the latest release. I had the same problem but after updating to the latest version, it passed.

• https://github.com/robotframework/robotframework/issues/3862

Could we have a fix for that or at least a workaround ?

13:13:10  + pabot --pabotlib --resourcefile AndroidDevicesSet.dat --verbose --processes 3 -v is_parallel:True -v app_location:****/apk_production_develop -v environment:prod -v platform:Android -v devices:AndroidBsDeviceSpec --exclude iOSOnly --skip Skip -v Headless_Mode:true -F robot -d TestResults --nostatusrc --runemptysuite --output output2.xml -R TestResults/output1.xml Robot/TestSuite/temp
13:13:11  /usr/local/lib/python3.9/site-packages/robot/utils/argumentparser.py:77: UserWarning: ArgumentParser option 'auto_pythonpath' is deprecated since Robot Framework 5.0.
13:13:11    warnings.warn("ArgumentParser option 'auto_pythonpath' is deprecated "
13:13:11  Storing .pabotsuitenames file
13:13:11  Suite names resolved in 0.8578720092773438 seconds
13:13:11  2023-05-03 10:13:11.439867 [PID:59589] [0] [ID:0] EXECUTING PARALLEL Temp with command:
13:13:11  robot --extension robot --suite Temp --exclude iOSOnly --rerunfailed TestResults/output1.xml --runemptysuite --skip Skip --variable is_parallel:True --variable app_location:****/apk_production_develop --variable environment:prod --variable platform:Android --variable devices:AndroidBsDeviceSpec --variable Headless_Mode:true --variable CALLER_ID:313c71c6b4a9447a8e1903c8a7f707ca --variable PABOTLIBURI:127.0.0.1:8270 --variable PABOTEXECUTIONPOOLID:0 --variable PABOTISLASTEXECUTIONINPOOL:1 --variable PABOTNUMBEROFPROCESSES:3 --variable PABOTQUEUEINDEX:0 --variable PABOTLASTLEVEL: --outputdir TestResults/pabot_results/0 --output output2.xml --listener RobotStackTracer --log NONE --report NONE --xunit NONE --consolecolors off --consolemarkers off Robot/TestSuite/temp
13:13:13  2023-05-03 10:13:12.512947 [PID:59589] [0] [ID:0] PASSED Temp in 1.0 seconds
13:13:13  ==============================================================================
13:13:13  Temp                                                                          
13:13:13  ==============================================================================
13:13:13  Temp                                                                  | SKIP |
13:13:13  0 tests, 0 passed, 0 failed
13:13:13  ==============================================================================
13:13:13  Output:  /Users/jive/jenkins/prod/workspace/dev-gtc-mobile/goto-mobile-robot-e2e/packages/robot-framework/TestResults/pabot_results/0/output2.xml
13:13:13  
13:13:13  
13:13:13  0 tests, 0 passed, 0 failed, 0 skipped.
13:13:13  ===================================================
13:13:13  Output:  /Users/jive/jenkins/prod/workspace/dev-gtc-mobile/goto-mobile-robot-e2e/packages/robot-framework/TestResults/output2.xml
13:13:13  [ ERROR ] Suite 'Temp' contains no tests after model modifiers.

[efrenette11]

Here's what we have when using robot without pabot

13:42:29  + python3 -m rob
ot -v app_location:****/apk_production_develop -v environment:prod -v platform:Android -v devices:AndroidBsDeviceSpec --exclude iOSOnly --skip Skip -v Headless_Mode:true -F robot -d TestResults --nostatusrc --runemptysuite --output output3.xml -R TestResults/output2.xml Robot/TestSuite/temp
13:42:29  ==============================================================================
13:42:29  Temp                                                                          
13:42:29  ==============================================================================
13:42:29  Temp                                                                  | SKIP |
13:42:29  0 tests, 0 passed, 0 failed
13:42:29  ==============================================================================
13:42:29  Output:  /Users/jive/jenkins/prod/workspace/dev-gtc-mobile/goto-mobile-robot-e2e/packages/robot-framework/TestResults/output3.xml
13:42:29  Log:     /Users/jive/jenkins/prod/workspace/dev-gtc-mobile/goto-mobile-robot-e2e/packages/robot-framework/TestResults/log.html
13:42:29  Report:  /Users/jive/jenkins/prod/workspace/dev-gtc-mobile/goto-mobile-robot-e2e/packages/robot-framework/TestResults/report.htm

[patrijua]

Stumbled into same issue. Since RF can handle the same without error, so should pabot.