How to access CLI parameters when running with xdist

[jcalienni]

Hello, so i've been using sys.argv to know wich environment the tests should run form the CLI and depending on the environment the data the tests use.

tests/data.py

def env():
if "--env=DEV" in sys.argv:
    return 'DEV'
elif "--env=UAT" in sys.argv:
    return 'UAT'
elif "--env=PROD" in sys.argv:
    return 'PROD'
elif "--env=TE" in sys.argv:
    return 'TE'
elif "--env=PERF" in sys.argv:
    return 'PERF'
else:
    return 'Incorrect environment'

if env() == 'UAT':
     policy_list_fnol_ap = ['30-60000550', '30-60000551', '30-60000555', '30-60000727', '30-60000798', '30-60000549']
     policy_list_fnol_auto = ['4-65000343', '44-45000075', '44-45000076', '4-65000344']
     policy_list_fnol_vida = ['23-54000022', '23-54000023', '17-55000042']
elif env() == 'DEV':
     policy_list_fnol_ap = ['30-60000301', '30-60000302', '30-60000303', '30-60000304', '30-60000305', '30-60000300']
     policy_list_fnol_auto = ['4-65000178', '44-45000030', '4-65000179', '44-45000031']
     policy_list_fnol_vida = ['23-54000020', '23-54000021', '17-55000018']

If i run this ussing "-n" parameter, sys.argv returns "-c" parameter so i can't get the environment from CLI with -n. I was reading the documentation:

Accessing sys.argv from the master node in workers To access the sys.argv passed to the command-line of the master node, use request.config.workerinput["mainargv"].

I'm not sure how to implement this. My problem is that this code isn't inside the tests or conftest.py, is just a data.py file that defines the data and then the data is used as paramteres:

policy_list = tests.data.policy_list_fnol_ap
@pytest.mark.parametrize('policy_number', policy_list)
def test_fnol_delete_draft:
    script....

Is there a way i can access the CLI parameters outside the tests? With sys.argv i'm able to do it, but sys.argv doesn't work with xdist since the response is "-c" and not the full CLI.

Thanks

[RonnyPfannschmidt]

You can't, please use the pytest mechanism to register cli args and the pytest configuration to access them

[jcalienni]

You can't, please use the pytest mechanism to register cli args and the pytest configuration to access them

Thanks for the answer. I'm ussing fixtures to get the environment from the cli args and using it for the tests, but i don't know how to use that fixture in data.py since it isn't a test, is just a file with data.

Any advice if there is a better way to do this? I can run the test in pararell without problems if the data set is the same for all environments since i don't need to know the environment. But since each environment has their own set of data i would need to get that information to specify with data set to use.

[RonnyPfannschmidt]

@jcalienni typically the suggestion is to have a hook that applies parametrize to a fixture

aka first step - don't use any random globals and/or ensure you run a configure function from the pytest_configure hook as you set up your test env

so there would be a pytest_addoption(parsers) hook to add the new option and a pytest_configure hook to configure your modules for it

[jcalienni]

@jcalienni typically the suggestion is to have a hook that applies parametrize to a fixture

aka first step - don't use any random globals and/or ensure you run a configure function from the pytest_configure hook as you set up your test env

so there would be a pytest_addoption(parsers) hook to add the new option and a pytest_configure hook to configure your modules for it

Thanks for the answer, and sorry for all the questions,

This is what i have in conftest regarding environment variable.

@pytest.fixture()
def environment(env):
    if env in ENVIRONMENTS:
        return env
    elif env is None:
        raise Exception("Must specify environment in CLI, use one this options --env==('DEV', 'TE', 'PROD', 'PERF', 'UAT')".format(env))
    else:
        raise Exception("There is no '{}' environment, use this options ('DEV', 'TE', 'PROD', 'PERF', 'UAT')".format(env))

@pytest.fixture()
def env(request):  # retorna el valor del environment al setup
    return request.config.getoption("--env")

def pytest_addoption(parser): 
    parser.addoption("--env")

So i use "environment" fixture in tests to know wich environment should they run.

Now when you say "and a pytest_configure hook to configure your modules for it" i never used "pytest_configure". I'm now reading documentation to know what is the use for this function to see if this will help to get the environment variable outside a test so i can use it in data.py

Thanks again.

[RonnyPfannschmidt]

i glad you where able to solve the first part it with little instruction (im short on time atm)

for better handing of whats supported you can consider some extra details that can hep

a) pytests UsageError which hels

b) using a choice option to make it safe at cli parsing time

for pytest_configure - in that hook you can basically read the environment value and add a pytest hook to put the right parameters into fixtures/parameterize calls

an example of a plugin that configures a additional plugin when enabled can be seen under

https://github.com/pytest-dev/pytest/blob/c94766a580d10449536c75fbbdb4d9e9f3ba2082/src/_pytest/junitxml.py#L426-L440

a fun fact is - such a plugin class can have fixtures, and you can configure those fixtur parameters at setup time

so for the parameter sets you tend to use, you could inside of that function declare a correctly configured fixture

[jcalienni]

i glad you where able to solve the first part it with little instruction (im short on time atm)

Actually i allready had that code since i've been using the enviromnment fixture for the tests without problems, my problem starts trying to get the environment value in data.py since it's not a test, it's just a .py file with data, so i can't use the environment fixture to know wich set of data the test should use.

for better handing of whats supported you can consider some extra details that can hep

a) pytests UsageError which hels

b) using a choice option to make it safe at cli parsing time

for pytest_configure - in that hook you can basically read the environment value and add a pytest hook to put the right parameters into fixtures/parameterize calls

an example of a plugin that configures a additional plugin when enabled can be seen under

https://github.com/pytest-dev/pytest/blob/c94766a580d10449536c75fbbdb4d9e9f3ba2082/src/_pytest/junitxml.py#L426-L440

a fun fact is - such a plugin class can have fixtures, and you can configure those fixtur parameters at setup time

so for the parameter sets you tend to use, you could inside of that function declare a correctly configured fixture

I've spent all day reading about fixtures, plugins and hooks but still wasn't able to do it. I can't find a way to get the environment value in data.py when runing with xdist.

This is my data.py, when i try to get the environment using fixture i can't reuse the function to define with data set use since fixtures are not suposed to be used in function like this and i can't pass a parameter to env()

import random
import sys
import pytest

#Try to get environment value with fixture
def env(environment):
    return environment

if env() == 'UAT':
    policy_list_fnol_ap = ['30-60000550', '30-60000551', '30-60000555', '30-60000727', '30-60000798', '30-60000549']
    policy_list_fnol_auto = ['4-65000343', '44-45000075', '44-45000076', '4-65000344']
    policy_list_fnol_vida = ['23-54000023']
    policy_list_fnol_delete_draft = ['23-54000023']

    policy_list_notes = ['4-65000343', '44-45000075', '30-60000555', '23-54000023']
    claim_list_notes = ['30-3000117', '4-3000488', '23-3000015']

    policy_list_documents = [random.choice(policy_list_notes)]
    claim_list_documents = [random.choice(claim_list_notes)]

elif env() == 'DEV':
    policy_list_fnol_ap = ['30-60000301', '30-60000302', '30-60000303', '30-60000304', '30-60000305', '30-60000300']
    policy_list_fnol_auto = ['44-45000030']
    policy_list_fnol_vida = ['23-54000020', '23-54000021', '17-55000018']
    policy_list_fnol_delete_draft = ['44-45000030']

    policy_list_notes = ['4-65000178', '44-45000030', '30-60000301', '23-54000020']
    claim_list_notes = ['23-3000007', '4-3000178', '30-3000068']

    policy_list_documents = [random.choice(policy_list_notes)]
    claim_list_documents = [random.choice(claim_list_notes)]    

else:
    raise Exception("There is no test data for this environment, look in data.py. {}".format(sys.argv))

I'm pretty sure my take is not the best way to define wich data set to use.

Anyways enough for today, weekend time :)

Thanks a lot and i will keep trying monday, hopefully i will make it work so i can run tests with xdist.

[jcalienni]

Ok i wansn't able to do this alone so i asked for help. This is what we did with the help of someone in my work who knows a lot of everything and it works perfectly with and without -n

def env():

    found = len(list(filter(lambda parm: match('--env=', parm), sys.argv))) != 0
    args = None

    if found:
        # No subprocess
        args = sys.argv
    else:
        # Process forked by xdist
        args = psutil.Process().parent().cmdline()

With this code i'm able to get the command line with and without xdist

[RonnyPfannschmidt]

that solution works only for the simple case and is outside of what you "should" do with pytest we should add a documentation example on how to use and pass cli options for pytest correctly