pytest-dev / pytest

The pytest framework makes it easy to write small tests, yet scales to support complex functional testing
https://pytest.org
MIT License
11.98k stars 2.66k forks source link

pytest_generate_tests not called for tests in symlinked directory #5251

Open Drau opened 5 years ago

Drau commented 5 years ago

When symlinks exist in tests directory, the tests inside those files do not have access to fixtures created by pytest_generate_tests

Env: platform linux -- Python 3.6.6, pytest-4.5.0, py-1.7.0, pluggy-0.11.0

Given the fallowing File Tree:

root
|-- conftest.py
|-- real_tests <---- Actual directory containing test files
    |-- test_1.py 
|-- sym_tests <------ symlink directory to a directory containing test files
    |-- test_2.py

and the fallowing conftest.py:

import pytest

def pytest_addoption(parser):
    parser.addoption("--job_id", default=None, help="Job ID")

def pytest_generate_tests(metafunc):
    if 'job_id' in metafunc.fixturenames:
        metafunc.parametrize("job_id", [metafunc.config.option.job_id], scope='session')

test_1.py

import pytest

def test_1(job_id):
    print(f'Passing:  {job_id}')
    assert True

test_2.py

import pytest

def test_2(job_id):
    print(f'Passing:  {job_id}')
    assert True

Running 'pytest -s - - job_id 10' fails on test_2 with: fixture 'job_id' not found

blueyed commented 5 years ago

Where's the conftest?

Drau commented 5 years ago

Just above test_1.py :)

blueyed commented 5 years ago

In root? Otherwise it's a matter of contest only being used for files below it (and therefore not related to symlinks).

blueyed commented 5 years ago

btw: check out tree for displaying files in a tree-like fashion.

Drau commented 5 years ago

I've just realized what you were asking :) Yes, conftest is located in root, I've updated the comment.

If I remove sym_tests symlink, all works perfectly.

EDIT: I see now that pytest_generate_tests is not actually called for tests located in the symlink directory. But if I define some random fixture in that same conftest file, it is available for tests located in the symlink directory.

blueyed commented 5 years ago

If I remove sym_tests symlink, all works perfectly.

You mean if you make it a directory instead?

I see now that pytest_generate_tests is not actually called for tests located in the symlink directory.

Good find already, and that might be the issue then after all.

Would be great if you could turn this into a (failing) test for pytest already, using testdir / runpytest to show this.

blueyed commented 5 years ago

Also check out --collect-only and/or add prints into hooks to see what is going on.