Pabot evaluates tags before applying pre-run modifiers to test suites

[Serhiy1]

I have a pre-run modifier that adds empty test stubs to all suites, so I can automatically exercise the suite setups.

class PreRunModifier(SuiteVisitor):

    def start_suite(self, suite):
        front_load: TestCase = suite.tests.create(
        name="Front load users",
        tags=["FrontLoad"],
        doc="Test stub to only run the suite setup")

        front_load.body.create_keyword("Log", args=[suite.name])

As you can see, the test case does not exist until the pre-run modifier is run.

---

Directly running from robot, the test cases are successfully identified and run

robot --include FrontLoad --argumentfile work/debug.txt --prerunmodifier ./prerun_modifiers/PreRunModifier.py  ./archivist/

==============================================================================
Archivist :: util keywords that run on the startup of robot                   
...
93 tests, 93 passed, 0 failed
==============================================================================
Output:  /workspaces/avid/testing/robot/tests/output.xml
Log:     /workspaces/avid/testing/robot/tests/log.html
Report:  /workspaces/avid/testing/robot/tests/report.html

Running with pabot is another story, as it fails to find any tests

pabot --pabotlib --processes 4 --include FrontLoad --argumentfile work/debug.txt --prerunmodifier ./prerun_modifiers/PreRunModifier.py  ./archivist/ 
Robot Framework remote server at 127.0.0.1:8270 started.
Suite 'Archivist' contains no tests matching tag 'FrontLoad'.
Stopping PabotLib process
Robot Framework remote server at 127.0.0.1:8270 stopped.
PabotLib process stopped
Total testing: 0.0 seconds
Elapsed time:  1.48 seconds

[mkorpela]

So the problem is that the tag is not seen? Has this worked before? I think dynamic tags currently do not work in general because they can not be analysed before execution.

[Serhiy1]

Yeah, as I understand, the problem is that the tag is not seen. I haven't tested it on any other versions of pabot, but If this kind of behaviour were not considered in the design of pabot, I would wager that the pre-run modifier wouldn't work with any version.

However, the pre-run modifier does work with vanilla robot, and I think the use case I presented to need this capability is not unreasonable.

[j0hnrj]

This applies to when we use "--test" flag as well. Pre run modifiers assist a lot in setting up things globally before executions are done (certain things can't be put in suite setups). Can this be fixed in any of the future releases as it is breaking our scripts when we run in parallel with test flag enabled?