pytest-dev / pytest

The pytest framework makes it easy to write small tests, yet scales to support complex functional testing
https://pytest.org
MIT License
12.13k stars 2.68k forks source link

strange handling of test ids for parametrize #6363

Open graingert opened 4 years ago

graingert commented 4 years ago
$ cat tests/test_toast.py
import pytest

@pytest.mark.parametrize(
    "ham,spam",
    [
        ("/bacon/../", u"//bacon/../"),
        ("/bacon/../../", u"//bacon/../../"),
        ("/bacon//", u"//bacon//"),
    ],
)
def test_eggs(ham,spam):
    pass
$ pytest -vvv tests/test_eggs.py
==================================== test session starts ====================================
platform linux -- Python 3.7.5, pytest-5.3.2, py-1.8.0, pluggy-0.12.0 -- /home/graingert/.virtualenvs/pytest/bin/python3.7
cachedir: .pytest_cache
rootdir: /home/graingert/projects/sausage/tests, inifile: pytest.ini
collected 3 items                                                                           

tests/test_toast.py::test_eggs[/-/] PASSED                                            [ 33%]
tests/] PASSED                                                                        [ 66%]
tests/test_toast.py::test_eggs[/bacon/-/bacon/] PASSED                                [100%]

===================================== 3 passed in 0.01s =====================================
graingert commented 4 years ago

of particular concern is ("/bacon/../../", u"//bacon/../../") -> tests/] PASSED

graingert commented 4 years ago

this also happens on pytest==4.6.5

Zac-HD commented 4 years ago

FYI you can syntax-highlight code blocks.

Not sure what's happening here but I agree it's not good!

graingert commented 4 years ago

FYI you can syntax-highlight code blocks.

I know but GitHub bash syntax highlighting is not smart enough to detect cat and file extensions

asottile commented 4 years ago

if you use


    ```console
    $ foo
    bar

it will highlight as

```console
$ foo
bar
blueyed commented 4 years ago

@graingert Not sure what the problem here is, but it looks like this for me, i.e. the params are joined using "-" (which appears to not be the case for you I guess?):

============================= test session starts ==============================
platform linux -- Python 3.8.1, pytest-5.3.3.dev46+g83ba709c4.d20200112, py-1.8.1, pluggy-0.13.2.dev5+g76838b1 -- …/Vcs/pytest/.venv/bin/python
cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('…/Vcs/pytest/.hypothesis/examples')
rootdir: …/Vcs/pytest, inifile: tox.ini
plugins: hypothesis-5.1.2
collecting ... collected 3 items

t/t_param.py::test_eggs[/bacon/../-//bacon/../] PASSED                   [ 33%]
t/t_param.py::test_eggs[/bacon/../../-//bacon/../../] PASSED             [ 66%]
t/t_param.py::test_eggs[/bacon//-//bacon//] PASSED                       [100%]

============================== 3 passed in 0.03s ===============================
blueyed commented 4 years ago

You have

cat tests/test_toast.py

and

pytest -vvv tests/test_eggs.py

Are symlinks involved maybe?

blueyed commented 4 years ago

I can reproduce it after touch tests/pytest.ini / touch t/pytest.ini.. :+1:

graingert commented 4 years ago

@blueyed reproduction as xfail test here: https://github.com/pytest-dev/pytest/pull/6394