pabot fails with most-recent robot 5.0 beta build - invalid option pythonpath

[oboehmer]

pabot (using up-to-date 1.13 version) doesn't work with robotframework 5.0 beta1:

root@acd2910875f4:/# pabot workspace/test.robot
Invalid option 'pythonpath'.
Total testing: 0.0 seconds
Elapsed time:  0.0 seconds

This looks to be a consequence of the fix for issue https://github.com/robotframework/robotframework/issues/4212 which introduced a validation of the args passed into robot.run (And the equivalent rebot api). The pythonpath option is not supported in robot.run api as documented in https://github.com/robotframework/robotframework/blob/94e1206bf259bae4c752b5a19d6f875c63934e82/src/robot/run.py#L502-L504

[aaltat]

I see the same thing, but i have more invalid options: invalid options 'skip', 'skiponfailure', 'variable', 'variablefile', 'listener', 'prerunmodifier' and 'pythonpath'. List is from https://github.com/MarketSquare/robotframework-browser atest.

[mkorpela]

Seems that this points to https://github.com/robotframework/robotframework/blob/fd3ef5a3d0f472d3f2e08f1f38a6f44a02086a2e/src/robot/conf/settings.py#L87

[mkorpela]

Those options are used for suite dynamic generation https://github.com/mkorpela/pabot/blob/eaa0dd3a25e191db13126699b0c8093b81fe36e9/pabot/pabot.py#L1072

[mkorpela]

The opts == options come from

options, datasources = ArgumentParser(
        USAGE,
        auto_pythonpath=False,
        auto_argumentfile=True,
        env_options="ROBOT_OPTIONS",
    ).parse_args(args)

[pekkaklarck]

As @oboehmer already pointed out, this seems to be caused by https://github.com/robotframework/robotframework/issues/4212. I hope this can be handled on Pabot side nicely. If not, we need to consider reverting the change on RF side.

Where does that USAGE in your example originate from and what does it contain @mkorpela? If it's created on Pabot side, hopefully it would be enough to remove options that aren't anymore supported by RF 5.0 i.e. --critical, --noncritical and --xunitskipnoncritical.

[mkorpela]

from robot.run import USAGE

[pekkaklarck]

I have done changes on Robot side to help mitigate problems caused by https://github.com/robotframework/robotframework/issues/4212 and also studied what changes would be needed on Pabot side. It seems only two Pabot changes are needed and that they are compatible also with earlier RF versions. I explain the changes in separate comments below.

[pekkaklarck]

Problem 1. Pabot's merge functionality creates RebotSettings using options parsed against Robot's execution side parameters which nowadays causes an error. The code is here: https://github.com/mkorpela/pabot/blob/main/pabot/result_merger.py#L248

The problem can be fixed by changing the current

settings = RebotSettings(options)

to

settings = RobotSettings(options).get_rebot_settings()

With RF 5.0 beta 1 get_rebot_settings() returns settings that aren't supported by Rebot anymore, but because the only settings usages are settings.output_directory, settings.critical_tags and settings.non_critical_tags that problem doesn't matter here.

The aforementioned settings usages related to critical tags are only needed to support RF < 4. If Pabot would have RF 4.0 as the minimum version, then the only settings usage would be settings.output_directory. Then it could be better to change the merge() function to accept output_directory instead of options. Neither RobotSettings or RebotSettings wouldn't be needed here at all after that.

[pekkaklarck]

Problem 2. Pabot creates options it uses with rebot() based on options parsed against Robot's execution side parameters. The code is here: https://github.com/mkorpela/pabot/blob/main/pabot/pabot.py#L1162

This problem could be fixed by creating a new options dictionary instead of copying all execution side parameters and modifying them. I changed the _options_for_rebot as below and was able to run Pabot using RF 5.0 beta 1. We'd still need to check what options to actually return but I believe this would be a good approach.

def _options_for_rebot(options_for_run, start_time_string, end_time_string):
    return {
        'starttime': start_time_string,
        'endtime': end_time_string,
        'metadata': options_for_run['metadata'],
        'tagstatinclude': options_for_run['tagstatinclude'],
        'tagstatexclude': options_for_run['tagstatexclude'],
        'tagstatcombine': options_for_run['tagstatcombine'],
        'tagdoc': options_for_run['tagdoc'],
        'tagstatlink': options_for_run['tagstatlink'],
        'expandkeywords': options_for_run['expandkeywords'],
        'removekeywords': options_for_run['removekeywords'],
        'flattenkeywords': options_for_run['flattenkeywords'],
        'prerebotmodifier': options_for_run['prerebotmodifier'],
    }

[oboehmer]

Hi @mkorpela ! Thanks for addressing this issue. When do you plan to release the fix to validate it with 5.0rc1?

thanks a lot for pabot!