Test cases are parsed by option --testlevelsplit to be repeated

[wendi616]

Hi Pabot Team,

I found an issue when using pabot.

In my work scenario, I want to specify the cases which are expected to execute, and included them in an argument file. And at the mean time, I want to split them as test level, so the option --testlevelsplit was also appended. And a listener is added to monitor if the cases are executed as expected. The final command can be found as below.

wrong command

pabot --testlevelsplit -A arg_case.txt --listener test.py .

correct command

pabot -A arg_case.txt --listener test.py .

And then when I check the listener log, the cases are executed repeatly.

If remove the option --testlevelsplit, the issue will not occur.

So I tried another scenario, to specify the suites list. And found if --testlevelsplit option is appended, everything will work fine. But the same issue will occur if remove this option.

wrong command

pabot -A arg_suite.txt --listener test.py .

correct command

pabot --testlevelsplit -A arg_suite.txt --listener test.py .

Would you please help check if they are correct? According to the documentation, this option only declare the exectuion level, will not affect the number of executions, correct?

All test data can be found in attachment.

env info:

pip list Package Version

---

natsort 8.2.0 natsort 8.2.0 pip 22.3.1 robotframework 6.0.2 robotframework-pabot 2.12.0

python --version Python 3.11.1

Thank you!

BR Wendy pabot_test.zip

[xtergo]

I'm also affected by this problem. Would be good to get some feedback on this!

[bli74]

I think, the selection of tests and suites in an argument file does not fit to the design of pabot:

pabot main process determines the list of suites (or tests in case of --testlevelsplit) and passes them on command line as '--suite suiteName' or '--test testName' to the pabot execution processes. The argument file is passed to the execution process as it is, e.g. to set common variables. Parameters '--suite' and '--test' from argument file are not filtered out to run only the suite or test specified on command line.

The correct way is to specify all '--suite' and '--test' parameters on pabot command line and not in an argument file. Then the single pabot execution processes run just the suite or test, for which they were started.

[Aadil-5122]

Hello,

I am new to this repo and have been working on this issue. I tried to test arg_case.txt locally but kept encountering errors. I edited the file and tested it both with and without the --testlevelsplit flag. In both cases, I received the same result: no repeated execution of test cases. If that is the issue, it might be a special case where this result is being obtained.

Additionally, I noticed that if I provide a path, all the Robot files in that path get executed, even if they are not listed in the argument file. If this is the issue you are facing, I can work on that.

For clarity, I am attaching two screenshots along with the edited arg_case.txt file.

Files attached:

• arg_case.txt
• Screenshot 1
• Screenshot 2

arg_case.txt