I can run a Robot DataDriver testsuite file by itself in Pabot with multiple processes, and with other robot files in shards, but if I try to run it by itself in shards I am told it only has 1 testcase which is not enough for a 2 or more shards.
That is easy to work around, but the DataDriver testsuites are not broken up between the shards causing some shards to take far longer to run (i.e. the number of testcases is really unbalanced)
The example contains 2 suites: Pabot_Test.robot which uses an Excel file with the DataDriver and is ...reused... from someone-else's example, while invalid_login.robot is a gutted version of the templated web demo code.
Running Pabot with Pabot_Test.robot on a single machine works fine - so there are no issues with tags etc.
(venv) PS C:\Users\burrk\VSCodeProjects\Test> pabot --testlevelsplit --pabotlib .\Test\Pabot_Test.robot
Robot Framework remote server at 127.0.0.1:8270 started.
2022-06-28 14:42:42.899869 [PID:19704] [0] [ID:0] EXECUTING Pabot Test.Verify Most Searched Term ${items}
2022-06-28 14:42:45.428317 [PID:19804] [3] [ID:4] EXECUTING Pabot Test
2022-06-28 14:42:45.428317 [PID:15108] [2] [ID:1] EXECUTING Pabot Test
2022-06-28 14:42:45.428317 [PID:15688] [1] [ID:2] EXECUTING Pabot Test
[...lines removed...]
2022-06-28 14:43:06.546313 [PID:8436] [0] [ID:24] PASSED Pabot Test in 8.2 seconds
2022-06-28 14:43:06.565220 [PID:17908] [5] [ID:23] PASSED Pabot Test in 8.0 seconds
2022-06-28 14:43:10.002376 [PID:21796] [2] [ID:25] PASSED Pabot Test in 4.7 seconds
25 tests, 25 passed, 0 failed, 0 skipped.
===================================================
Output: C:\Users\burrk\VSCodeProjects\Test\output.xml
Log: C:\Users\burrk\VSCodeProjects\Test\log.html
Report: C:\Users\burrk\VSCodeProjects\Test\report.html
Stopping PabotLib process
Robot Framework remote server at 127.0.0.1:8270 stopped.
PabotLib process stopped
Total testing: 2 minutes 38.69 seconds
Elapsed time: 29.6 seconds
But running it as a shard makes Pabot claim it doesn't contain enough testcases to split!
(venv) PS C:\Users\burrk\VSCodeProjects\Test> pabot --testlevelsplit --shard 2/2 --pabotlib .\Test\Pabot_Test.robot
Robot Framework remote server at 127.0.0.1:8270 started.
Storing .pabotsuitenames file
Not enought items (1) for shard cound (2).
Stopping PabotLib process
Robot Framework remote server at 127.0.0.1:8270 stopped.
PabotLib process stopped
Total testing: 0.0 seconds
Elapsed time: 1.18 seconds
[P.S. there are a couple of spelling mistakes in the error message ("enought", "cound") ... ]
However, running the testcase with a non-DataDriver testcase works fine:
2022-06-28 14:34:46.090196 [PID:25076] [0] [ID:27] PASSED Suites.Pabot Test in 6.9 seconds
2022-06-28 14:34:46.090196 [PID:24688] [2] [ID:26] PASSED Suites.Pabot Test in 6.9 seconds
2022-06-28 14:34:49.037498 [PID:16872] [5] [ID:28] PASSED Suites.Pabot Test in 3.6 seconds
28 tests, 28 passed, 0 failed, 0 skipped.
===================================================
Output: C:\Users\burrk\VSCodeProjects\Test\output.xml
Log: C:\Users\burrk\VSCodeProjects\Test\log.html
Report: C:\Users\burrk\VSCodeProjects\Test\report.html
Stopping PabotLib process
Robot Framework remote server at 127.0.0.1:8270 stopped.
Total testing: 2 minutes 29.0 seconds
Elapsed time: 25.45 seconds
Nonetheless, the two shards do not have a close to even number of testcases between them: 4 vs 28, instead of 16 each.
Is this a known problem, or a mistake on my part?
Data-Driven_in_Parallel_2.zip
I can run a Robot DataDriver testsuite file by itself in Pabot with multiple processes, and with other robot files in shards, but if I try to run it by itself in shards I am told it only has 1 testcase which is not enough for a 2 or more shards.
That is easy to work around, but the DataDriver testsuites are not broken up between the shards causing some shards to take far longer to run (i.e. the number of testcases is really unbalanced)
From: https://robotframework.slack.com/archives/C7HKR2L6L/p1656441862740679
I have attached example code that was also used in https://robotframework.slack.com/archives/C7HKR2L6L/p1647606951367239?thread_ts=1647471796.103679&cid=C7HKR2L6L, except I updated all the libraries to the most recent when reusing this example.
The example contains 2 suites:
Pabot_Test.robotwhich uses an Excel file with the DataDriver and is ...reused... from someone-else's example, whileinvalid_login.robotis a gutted version of the templated web demo code.Running Pabot with
Pabot_Test.roboton a single machine works fine - so there are no issues with tags etc.[...lines removed...]
But running it as a shard makes Pabot claim it doesn't contain enough testcases to split!
[P.S. there are a couple of spelling mistakes in the error message ("enought", "cound") ... ]
However, running the testcase with a non-DataDriver testcase works fine:
and
[...]
Nonetheless, the two shards do not have a close to even number of testcases between them: 4 vs 28, instead of 16 each. Is this a known problem, or a mistake on my part? Data-Driven_in_Parallel_2.zip