mkorpela / pabot

Parallel executor for Robot Framework test cases.
https://pabot.org
Apache License 2.0
476 stars 152 forks source link

--ordering not recognized #421

Closed admorgan closed 2 years ago

admorgan commented 3 years ago

$ pabot --version A parallel executor for Robot Framework test cases. Version 2.1.0

Supports all Robot Framework command line options and also following options (these must be before normal RF options):

--verbose more output

--command [ACTUAL COMMANDS TO START ROBOT EXECUTOR] --end-command RF script for situations where pybot is not used directly

--processes [NUMBER OF PROCESSES] How many parallel executors to use (default max of 2 and cpu count)

--testlevelsplit Split execution on test level instead of default suite level. If .pabotsuitenames contains both tests and suites then this will only affect new suites and split only them. Leaving this flag out when both suites and tests in .pabotsuitenames file will also only affect new suites and add them as suite files.

--resourcefile [FILEPATH] Indicator for a file that can contain shared variables for distributing resources.

--pabotlib Start PabotLib remote server. This enables locking and resource distribution between parallel test executions.

--pabotlibhost [HOSTNAME] Host name of the PabotLib remote server (default is 127.0.0.1)

--pabotlibport [PORT] Port number of the PabotLib remote server (default is 8270)

--ordering [FILE PATH] Optionally give execution order from a file.

--suitesfrom [FILEPATH TO OUTPUTXML] Optionally read suites from output.xml file. Failed suites will run first and longer running ones will be executed before shorter ones.

--argumentfile[INTEGER] [FILEPATH] Run same suite with multiple argumentfile options. For example "--argumentfile1 arg1.txt --argumentfile2 arg2.txt".

Copyright 2019 Mikko Korpela - Apache 2 License

Robot Framework 4.1 (Python 3.8.10 on linux) Total testing: 0.0 seconds Elapsed time: 0.0 seconds $ pabot --version --ordering tests/tests.order option --ordering not recognized Total testing: 0.0 seconds Elapsed time: 0.0 seconds

NorseGaud commented 3 years ago

Seeing this with

option --processes not recognized
Total testing: 0.0 seconds
Elapsed time:  0.0 seconds

But it doesn't happen when I remove --loglevel DEBUG --listener RobotStackTracer.

I would remove --version and try it.

admorgan commented 3 years ago

Same result without the --version

NorseGaud commented 3 years ago

Is it possible that leaving off the actual path to your test is the issue? Complete the command: pabot --ordering tests/tests.order robot-tests/

mkorpela commented 3 years ago

Currently all pabot arguments must be before robot arguments. --version is robot argument and robot will not understand pabot arguments that come after that. PR that would make argument order free would be accepted.

shyam-somanathan commented 2 years ago

I’m facing a similar problem with pabot —ordering 
Using pabot v2.1.0 robot library - v4.1

My test suite folder 
 Features Feature1
          TestSuite1 (Multiple tests)
         |
          TestSuite2 (Multiple tests)
         |
          TestSuite3  (Multiple tests) 
         |
          TestSuite4  (Multiple tests)


Ordering file --test Features.Feature1.TestSuite1.Test1

WAIT

--suite Features.Feature1.TestSuite2

WAIT

--test Features.Feature1.TestSuite1.Test2

Iam running the tests as

pabot --ordering Tests/.pabotsuite_av Tests/Features

Instead of running only the tests and suite that I specified in the .pabotsuite_av , pabot seems to create instances for all test suites! Please correct me if iam wrong in the way I’m running pabot , or is this a known issue which is yet to be fixed ?

admorgan commented 2 years ago

Moving around the parameters I was able to get this execute.

shyam-somanathan commented 2 years ago

@admorgan Did the --ordering work as expected for you ? If so , please could you let me know the pabot command you used and the version of robot framework library that you are using . --ordering wasnt working the way it should be for me , with robot library - v4.1

admorgan commented 2 years ago

@shyam-somanathan, no the ordering did not work as expected. My original report was that the option was failing and I am past that. Now I am working up a sample to demonstrate what I expect and what I am actually seeing and will create a new ticket about the actual implementation.