arugment parsing issue

[jmccreight]

• [X] a detailed description of the bug or suggestion

There was an issue related to this but it was "closed". I dont see that the issue actually resolved. Even if this is a duplicate, I've found it extremely difficult to find any other matching issues here or elsewhere.... so it think it's worth repeating.

Pytest arugment parsing fails on the first absolute path argument when an = is not used. I assume the rules for failure are some how tied to rootdir identification because its then appears to be using the argument as root dir and coming up with the wrong conftest.py. Here's the simplest example:

(wrf_hydro_nwm_test) jamesmcc@cheyenne5[1080]:~/Downloads/testy> ls 
conftest.py  test_1.py
(wrf_hydro_nwm_test) jamesmcc@cheyenne5[1081]:~/Downloads/testy> cat conftest.py 
import pytest

def pytest_addoption(parser):
    parser.addoption(
        '--the_arg',
        required=True,
        action='store',
        help='domain directory'
    )

@pytest.fixture(scope="session")
def the_arg(request):
    return request.config.getoption("--the_arg")
(wrf_hydro_nwm_test) jamesmcc@cheyenne5[1082]:~/Downloads/testy> cat test_1.py 
def test_the_arg(the_arg):
    assert the_arg == 'foo'
(wrf_hydro_nwm_test) jamesmcc@cheyenne5[1083]:~/Downloads/testy> pytest --the_arg /
usage: pytest [options] [file_or_dir] [file_or_dir] [...]
pytest: error: unrecognized arguments: --the_arg
  inifile: None
  rootdir: /
(wrf_hydro_nwm_test) jamesmcc@cheyenne5[1084]:~/Downloads/testy> pytest --the_arg=/
=============================================== test session starts ================================================
platform linux -- Python 3.6.8, pytest-3.9.1, py-1.8.0, pluggy-0.12.0
rootdir: /glade/u/home/jamesmcc/Downloads/testy, inifile:
plugins: html-1.19.0, datadir-ng-1.1.0, metadata-1.8.0
collected 1 item                                                                                                   

test_1.py F                                                                                                  [100%]

===================================================== FAILURES =====================================================
___________________________________________________ test_the_arg ___________________________________________________

the_arg = '/'

    def test_the_arg(the_arg):
>       assert the_arg == 'foo'
E       AssertionError: assert '/' == 'foo'
E         - /
E         + foo

test_1.py:2: AssertionError
============================================= 1 failed in 0.07 seconds =============================================

• [X] output of pip list from the virtual environment you are using

(wrf_hydro_nwm_test) jamesmcc@cheyenne5[1085]:~/Downloads/testy> pip list
Package               Version    Location                                     
--------------------- ---------- ---------------------------------------------
atomicwrites          1.3.0      
attrs                 19.1.0     
backcall              0.1.0      
beautifulsoup4        4.8.0      
boltons               18.0.1     
bs4                   0.0.1      
certifi               2019.6.16  
cftime                1.0.3.4    
chardet               3.0.4      
Click                 7.0        
cloudpickle           1.2.1      
dask                  2.3.0      
DateTime              4.3        
decorator             4.4.0      
deepdiff              3.3.0      
distributed           2.3.2      
ESMPy                 7.1.0.dev0 
f90nml                1.0.2      
fsspec                0.4.4      
HeapDict              1.0.0      
idl-python            2.0        
idna                  2.7        
importlib-metadata    0.19       
ipython               7.8.0      
ipython-genutils      0.2.0      
jedi                  0.15.1     
jsonpickle            1.2        
locket                0.2.0      
matlabengineforpython R2018a     
more-itertools        7.2.0      
msgpack               0.6.1      
netCDF4               1.4.1      
numpy                 1.17.1     
pandas                0.23.4     
parso                 0.5.1      
partd                 1.0.0      
pathlib               1.0.1      
pexpect               4.7.0      
pickleshare           0.7.5      
pip                   19.2.3     
pluggy                0.12.0     
prompt-toolkit        2.0.9      
psutil                5.6.3      
ptyprocess            0.6.0      
py                    1.8.0      
pycodestyle           2.5.0      
Pygments              2.4.2      
pytest                3.9.1      
pytest-datadir-ng     1.1.0      
pytest-html           1.19.0     
pytest-metadata       1.8.0      
python-dateutil       2.8.0      
pytz                  2019.2     
PyYAML                5.1.2      
requests              2.20.0     
setuptools            41.0.1     
six                   1.12.0     
sortedcontainers      2.1.0      
soupsieve             1.9.3      
tblib                 1.4.0      
toolz                 0.10.0     
tornado               6.0.3      
traitlets             4.3.2      
urllib3               1.24.3     
wcwidth               0.1.7      
wheel                 0.33.4     
wrfhydropy            0.0.16     /glade/u/home/jamesmcc/WRF_Hydro/wrf_hydro_py
xarray                0.10.9     
zict                  1.0.0      
zipp                  0.6.0      
zope.interface        4.6.0      

• [X] pytest and operating system versions

For pytest, see both above.

(wrf_hydro_nwm_test) jamesmcc@cheyenne5[1086]:~/Downloads/testy> cat /etc/os-release
NAME="SLES"
VERSION="12-SP4"
VERSION_ID="12.4"
PRETTY_NAME="SUSE Linux Enterprise Server 12 SP4"
ID="sles"
ANSI_COLOR="0;32"
CPE_NAME="cpe:/o:suse:sles:12:sp4"

• [X] minimal example if possible See above.

[RonnyPfannschmidt]

I'm on mobile, but this looks like a classical case of non top level contest not partaking in cli parser configuration

The solution to this suspected case would be to have a top level contest or a entry point based plugin

[nicoddemus]

Hi @jmccreight,

@RonnyPfannschmidt is correct.

Here is what happens when the user executes pytest --the-arg /:

1. pytest starts up and all builtin and entry-point-based plugin options get registered.
2. Command-line is parsed: pytest sees ["--the-arg", "/"], so it parses:
   • --the-arg as <unknown option "--the-arg">
   • / as a <collection path "/">
3. It starts collecting "initial conftest" files, which are conftest.py files located either at the CWD (if no collection path(s) is given in the command-line), or at the paths given in the command-line. Here pytest assumes you meant to start collection from /.
4. It starts to collect from /, but it does not see any conftest files there, so the "initial conftest" collection stage ends. At this stage, if it had found conftest files, they would be processed, but none have been found so nothing happens.
5. It resumes processing the command-line, now trying to match previously unknown options to newly discovered flags; the conftest from CWD was not yet been processed, so --the-flag is not yet registered, so it errors out with an unknown flag.

But when the user executes pytest --the-arg=/:

1. The unknown option is now <unknown option "--the-arg=/">.
2. Because no <collection path> was given, it starts collecting from CWD.
3. The conftest.py file is found as an "initial conftest", so it is processed and --the-arg gets registered.
4. It resumes processing the command-line, and it successfully parses --the-arg=/.

The problem is that we can't know if a "path" lead by an unknown option will be consumed by it or not: the unknown option might have been just as well an option that does not consume the next argument, for example --enable. For example, in this command-line:

$ pytest --enable /

Probably means that --enable is a bool option and pytest should collect from /, but we can't really know that before hand.