Open dylan-at-na opened 2 months ago
I think this is better raised as a feature request at pylint-dev/pylint-pytest
Hello @dylan-at-na!
Are you sure this is working like you are demonstrating?
We do actually have a test like that in the repo: https://github.com/pylint-dev/pylint-pytest/blob/8310a88916e4b0b7462fc525c56d60b9af4dc126/tests/input/no-member/not_using_cls.py
I have tried to replicate your setup
$ git --no-pager diff
diff --git a/tests/input/no-member/not_using_cls.py b/tests/input/no-member/not_using_cls.py
index 95438af..16caf33 100644
--- a/tests/input/no-member/not_using_cls.py
+++ b/tests/input/no-member/not_using_cls.py
@@ -7,6 +7,14 @@ class TestClass:
def setup_class(request):
clls = request.cls
clls.defined_in_setup_class = 123
+ print(clls.defined_in_setup_class)
def test_foo(self):
assert self.defined_in_setup_class
+
+ @pytest.fixture(scope="class")
+ def setup_teardown_environment(self, request: pytest.FixtureRequest):
+ # The following line assigns a value to `temp1`
+ # which can be accessed throughout the test class via `self`.
+ request.cls.temp1 = 54321
+ print(self.temp1)
$ pytest tests/input/no-member/not_using_cls.py -s
======================================= test session starts =======================================
platform linux -- Python 3.11.9, pytest-7.4.3, pluggy-1.3.0 -- ~/pylint-pytest/.venv/bin/python3.11
cachedir: .pytest_cache
rootdir: ~/pylint-pytest
configfile: pyproject.toml
plugins: cov-5.0.0
collected 1 item
tests/input/no-member/not_using_cls.py::TestClass::test_foo 123
PASSED
======================================== 1 passed in 0.01s ========================================
but, as you can see, I am unable to make your fixture
work.
The test is printing out 123
(clls.defined_in_setup_class
), but not 54321
.
You MUST mark the fixture
as autouse=True
, according to the documentation: https://docs.pytest.org/en/latest/how-to/unittest.html#using-autouse-fixtures-and-accessing-other-fixtures
(Let's find out if the reporter is using pylint-pytest or not.)
@stdedos Please see the below example which demonstrates a functional example (autouse is ignored in the first example, but is used further down)...
import pytest
class TestExample:
@pytest.fixture(scope="class")
def setup_teardown_username(self, request: pytest.FixtureRequest):
request.cls.username = 'dylan'
yield True
del request.cls.username
@pytest.fixture(scope="function")
def setup_teardown_password(self, request: pytest.FixtureRequest, setup_teardown_username):
request.cls.password = '*****'
yield setup_teardown_username, True
del request.cls.password
def test_username_and_password(self, setup_teardown_password):
assert self.username == 'dylan'
assert self.password == '*****'
And the log output from running the above...
source .../.venv/bin/activate.csh
% (pycldqa-py3.11) % /usr/bin/env .../.venv/bin/python /u/dylanl/.vscode-server/extensions/ms-python.debugpy-2024.7.11591012-linux-x64/bundled/libs/debugpy/adapter/../../debugpy/launcher 55139 -- -m pytest /u/dylanl/repos/local/scratch/test_scratch.py
================================= test session starts ==================================
platform linux -- Python 3.11.4, pytest-8.2.2, pluggy-1.5.0
rootdir: /u/dylanl/repos/local
configfile: pyproject.toml
plugins: xdist-3.6.1
collected 1 item
../local/scratch/test_scratch.py . [100%]
================================== 1 passed in 0.04s ===================================
The following screenshot demonstrates the error being thrown...
@jacobtylerwalls The following is the result of running pylint against the above code with pylint-pytest plugin enabled. Relevant lines are marked with ***
(pycldqa-py3.11) % cd ...; /usr/bin/env .../.venv/bin/python /u/dylanl/.vscode-server/extensions/ms-python.debugpy-2024.7.11591012-linux-x64/bundled/libs/debugpy/adapter/../../debugpy/launcher 36313 -- -m pylint --load-plugins=pylint_pytest /u/dylanl/repos/local/scratch/test_scratch.py
************* Module test_scratch
/u/dylanl/repos/local/scratch/test_scratch.py:9:0: C0303: Trailing whitespace (trailing-whitespace)
/u/dylanl/repos/local/scratch/test_scratch.py:15:0: C0303: Trailing whitespace (trailing-whitespace)
/u/dylanl/repos/local/scratch/test_scratch.py:1:0: C0114: Missing module docstring (missing-module-docstring)
/u/dylanl/repos/local/scratch/test_scratch.py:3:0: C0115: Missing class docstring (missing-class-docstring)
/u/dylanl/repos/local/scratch/test_scratch.py:5:4: C0116: Missing function or method docstring (missing-function-docstring)
/u/dylanl/repos/local/scratch/test_scratch.py:11:4: C0116: Missing function or method docstring (missing-function-docstring)
/u/dylanl/repos/local/scratch/test_scratch.py:16:4: C0116: Missing function or method docstring (missing-function-docstring)
***/u/dylanl/repos/local/scratch/test_scratch.py:17:15: E1101: Instance of 'TestExample' has no 'username' member (no-member)
***/u/dylanl/repos/local/scratch/test_scratch.py:18:15: E1101: Instance of 'TestExample' has no 'password' member (no-member)
-----------------------------------
Your code has been rated at 0.00/10
If I now add autouse=True
to the fixtures and make the necessary changes we end up with the following code...
import pytest
class TestExample:
@pytest.fixture(scope="class", autouse=True)
def setup_teardown_username(self, request: pytest.FixtureRequest):
request.cls.username = 'dylan'
yield True
del request.cls.username
@pytest.fixture(scope="function", autouse=True)
def setup_teardown_password(self, request: pytest.FixtureRequest):
request.cls.password = '*****'
yield True
del request.cls.password
def test_username_and_password(self):
assert self.username == 'dylan'
assert self.password == '*****'
Which again, passes.
================================= test session starts ==================================
platform linux -- Python 3.11.4, pytest-8.2.2, pluggy-1.5.0
rootdir: /u/dylanl/repos/local
configfile: pyproject.toml
plugins: xdist-3.6.1
collected 1 item
../local/scratch/test_scratch.py . [100%]
================================== 1 passed in 0.06s ===================================
However, running PyLint on this produces the same errors (relevant lines marked with ***)...
pylint --load-plugins=pylint_pytest /u/dylanl/repos/local/scratch/test_scratch.py
************* Module test_scratch
/u/dylanl/repos/local/scratch/test_scratch.py:9:0: C0303: Trailing whitespace (trailing-whitespace)
/u/dylanl/repos/local/scratch/test_scratch.py:15:0: C0303: Trailing whitespace (trailing-whitespace)
/u/dylanl/repos/local/scratch/test_scratch.py:1:0: C0114: Missing module docstring (missing-module-docstring)
/u/dylanl/repos/local/scratch/test_scratch.py:3:0: C0115: Missing class docstring (missing-class-docstring)
/u/dylanl/repos/local/scratch/test_scratch.py:5:4: C0116: Missing function or method docstring (missing-function-docstring)
/u/dylanl/repos/local/scratch/test_scratch.py:11:4: C0116: Missing function or method docstring (missing-function-docstring)
/u/dylanl/repos/local/scratch/test_scratch.py:16:4: C0116: Missing function or method docstring (missing-function-docstring)
***/u/dylanl/repos/local/scratch/test_scratch.py:17:15: E1101: Instance of 'TestExample' has no 'username' member (no-member)
***/u/dylanl/repos/local/scratch/test_scratch.py:18:15: E1101: Instance of 'TestExample' has no 'password' member (no-member)
------------------------------------------------------------------
Your code has been rated at 0.00/10 (previous run: 3.75/10, -3.75)
Interestingly enough if I use your code sample located at https://github.com/pylint-dev/pylint-pytest/blob/8310a88916e4b0b7462fc525c56d60b9af4dc126/tests/input/no-member/not_using_cls.py
I don't get the error
************* Module test_scratch
/u/dylanl/repos/local/scratch/test_scratch.py:12:0: C0304: Final newline missing (missing-final-newline)
/u/dylanl/repos/local/scratch/test_scratch.py:1:0: C0114: Missing module docstring (missing-module-docstring)
/u/dylanl/repos/local/scratch/test_scratch.py:4:0: C0115: Missing class docstring (missing-class-docstring)
/u/dylanl/repos/local/scratch/test_scratch.py:7:4: C0116: Missing function or method docstring (missing-function-docstring)
/u/dylanl/repos/local/scratch/test_scratch.py:11:4: C0116: Missing function or method docstring (missing-function-docstring)
------------------------------------------------------------------
Your code has been rated at 2.86/10 (previous run: 0.00/10, +2.86)
This prompted me to attempt PyLint with @staticmethod
applied to my fixtures but this did not work.
In a last ditch effort I assigned request.cls
to clls
as you do above...
import pytest
class TestExample:
@pytest.fixture(scope="class", autouse=True)
def setup_teardown_username(self, request: pytest.FixtureRequest):
clls = request.cls
clls.username = 'dylan'
yield True
del request.cls.username
@pytest.fixture(scope="function", autouse=True)
def setup_teardown_password(self, request: pytest.FixtureRequest):
clls = request.cls
clls.password = '*****'
yield True
del request.cls.password
def test_username_and_password(self):
assert self.username == 'dylan'
assert self.password == '*****'
Again, this passes.
And it removes the error line for the class
scoped fixture (setup_teardown_username
), but not for the function
scoped fixture (setup_teardown_password
). Relevant line marked with ***.
************* Module test_scratch
/u/dylanl/repos/local/scratch/test_scratch.py:10:0: C0303: Trailing whitespace (trailing-whitespace)
/u/dylanl/repos/local/scratch/test_scratch.py:17:0: C0303: Trailing whitespace (trailing-whitespace)
/u/dylanl/repos/local/scratch/test_scratch.py:1:0: C0114: Missing module docstring (missing-module-docstring)
/u/dylanl/repos/local/scratch/test_scratch.py:3:0: C0115: Missing class docstring (missing-class-docstring)
/u/dylanl/repos/local/scratch/test_scratch.py:5:4: C0116: Missing function or method docstring (missing-function-docstring)
/u/dylanl/repos/local/scratch/test_scratch.py:12:4: C0116: Missing function or method docstring (missing-function-docstring)
/u/dylanl/repos/local/scratch/test_scratch.py:18:4: C0116: Missing function or method docstring (missing-function-docstring)
***/u/dylanl/repos/local/scratch/test_scratch.py:20:15: E1101: Instance of 'TestExample' has no 'password' member (no-member)
------------------------------------------------------------------
Your code has been rated at 2.00/10 (previous run: 2.86/10, -0.86)
(Incorrect assumption, see follow up comment for clarification) ~So it would seem that clls
is a somewhat special variable as far as PyLint is concerned? Though based on this search there doesn't seem to be much reference to it https://www.google.com/search?q=pylint+%22clls%22&client=firefox-b-d&sca_esv=41efd453413bdc83&sca_upv=1&ei=1kKOZrm0GdiB9u8P0YCu0A0&ved=0ahUKEwi59_Dag5yHAxXYgP0HHVGAC9oQ4dUDCA8&uact=5&oq=pylint+%22clls%22&gs_lp=Egxnd3Mtd2l6LXNlcnAiDXB5bGludCAiY2xscyIyCBAhGKABGMMESNw1UIEVWP4zcAF4AZABAJgBjwGgAegDqgEDMC40uAEDyAEA-AEBmAIFoAKBBMICChAAGLADGNYEGEfCAggQABiABBiiBMICCBAAGKIEGIkFwgIFECEYoAGYAwCIBgGQBgiSBwMxLjSgB6cG&sclient=gws-wiz-serp~
Quick follow up - it isn't clls
that resolves it in the class
scoped fixture. It is simply the act of assigning request.cls
to a variable. Any variable name will do.
import pytest
class TestExample:
@pytest.fixture(scope="class", autouse=True)
def setup_teardown_username(self, request: pytest.FixtureRequest):
mocha = request.cls
mocha.username = 'dylan'
yield True
del request.cls.username
@pytest.fixture(scope="function", autouse=True)
def setup_teardown_password(self, request: pytest.FixtureRequest):
clls = request.cls
clls.password = '*****'
yield True
del request.cls.password
def test_username_and_password(self):
assert self.username == 'dylan'
assert self.password == '*****'
Passes.
PyLint:
************* Module test_scratch
/u/dylanl/repos/local/scratch/test_scratch.py:10:0: C0303: Trailing whitespace (trailing-whitespace)
/u/dylanl/repos/local/scratch/test_scratch.py:17:0: C0303: Trailing whitespace (trailing-whitespace)
/u/dylanl/repos/local/scratch/test_scratch.py:1:0: C0114: Missing module docstring (missing-module-docstring)
/u/dylanl/repos/local/scratch/test_scratch.py:3:0: C0115: Missing class docstring (missing-class-docstring)
/u/dylanl/repos/local/scratch/test_scratch.py:5:4: C0116: Missing function or method docstring (missing-function-docstring)
/u/dylanl/repos/local/scratch/test_scratch.py:12:4: C0116: Missing function or method docstring (missing-function-docstring)
/u/dylanl/repos/local/scratch/test_scratch.py:18:4: C0116: Missing function or method docstring (missing-function-docstring)
***/u/dylanl/repos/local/scratch/test_scratch.py:20:15: E1101: Instance of 'TestExample' has no 'password' member (no-member)
------------------------------------------------------------------
Your code has been rated at 2.00/10 (previous run: 2.00/10, +0.00)
Again this does not resolve the issue in the function
scoped fixture.
So it would seem that
clls
is a somewhat special variable as far as PyLint is concerned? ...
FWIW: Your search result gives exactly one result for me
clls
name is not "special" per-se; any name would do: https://github.com/pylint-dev/pylint-pytest/blob/8310a88916e4b0b7462fc525c56d60b9af4dc126/pylint_pytest/checkers/class_attr_loader.py#L42
... but for now, it MUST be like that.
I am not sure why the original author chose to write this test (clls.defined_in_setup_class = 123
) and skipped the most obvious request.cls.X = Y
(coming in a commit soon).
It is an issue with "contrived/MWE" examples that you cannot see "where" are they coming from.
Quick follow up - it isn't
clls
that resolves it in theclass
scoped fixture. It is simply the act of assigningrequest.cls
to a variable. Any variable name will do.
Exactly 🙃
@pytest.fixture(scope="function")
I will try this locally too. However, as of now, for a request.cls
to be considered eligible, it needs to satisfy https://github.com/pylint-dev/pylint-pytest/blob/8310a88916e4b0b7462fc525c56d60b9af4dc126/pylint_pytest/utils.py#L83-L107
I will try all scopes and see what their result is. And then code/test appropriately.
... it seems that this is getting quick traction, so I'll push a branch within the day.
One thing that I haven't been able to figure out, unfortunately, is that the pylint/pylint-pytest is sensitive to ordering.
tl;dr:
class TestClass:
@staticmethod
@pytest.fixture(scope="class", autouse=True)
def setup_class(request):
clls = request.cls
clls.defined_in_setup_class = 123
def test_foo(self):
assert self.defined_in_setup_class
works,
class TestClass:
def test_foo(self):
assert self.defined_in_setup_class
@staticmethod
@pytest.fixture(scope="class", autouse=True)
def setup_class(request):
clls = request.cls
clls.defined_in_setup_class = 123
fails.
astroid
visits top-to-bottom. In order to fix this "correctly" we MUST first visit eligible @pytest.fixture(autouse=True)
fixtures.
Perhaps after a possible __init__
?
Maybe even strictly in the correct call order (session, package, module, class, function
)
For anybody else who comes across this issue - the easiest temporary workaround with minimal changes to your code is to introduce a fixture which is class
scoped, and autouse
and initializes all relevant variables to None
within your request.cls
. See below for an example relevant to the above code snippet...
import pytest
class TestExample:
@pytest.fixture(scope='class', autouse=True)
def setup_teardown_request_cls(self, request: pytest.FixtureRequest):
_cls = request.cls
_cls.username = None
_cls.password = None
@pytest.fixture(scope="class")
def setup_teardown_username(self, request: pytest.FixtureRequest):
request.cls.username = 'dylan'
yield True
del request.cls.username
@pytest.fixture(scope="function")
def setup_teardown_password(self, request: pytest.FixtureRequest, setup_teardown_username: bool):
request.cls.password = '*****'
yield True
del request.cls.password
def test_username_and_password(self, setup_teardown_password: bool):
assert self.username == 'dylan'
assert self.password == '*****'
This has the added benefit of isolating the initialization, thus being easy to remove when and if @stdedos changes are in place. You can of course initialize your variables to any default value you like but for demonstrations sake I have used None
.
Bug description
Command used
Pylint output
Expected behavior
PyLint should not identify this as a problem at all, as request.cls and self are intrinsically linked when operating within the same test class.
Pylint version
OS / Environment
Ubuntu 20