High memory usage if tests called by test case name

[fontealpina]

I'm experiencing an high memory usage if the tests to run are called by test case name instead of test file name.
To explain my issue I will use the following trivial test, called test_try.py:

class TestSuite:
    def test_case_1(self):
        assert True

    def test_case_2(self):
        assert True

    def test_case_3(self):
        assert True

If I want to run all the tests, I can type:

$ pytest -v test_try.py
======================== test session starts ========================
collected 3 items

test_try.py::TestSuite::test_case_1 PASSED
test_try.py::TestSuite::test_case_2 PASSED
test_try.py::TestSuite::test_case_3 PASSED
====================== 3 passed in 0.01 seconds ====================

But in my workflow may be useful call the tests by test case name, so:

$ pytest -v test_try.py::TestSuite::test_case_1 test_try.py::TestSuite::test_case_2 test_try.py::TestSuite::test_case_3
======================== test session starts ========================
collected 12 items

NuSMV/test/bug_026/test_try.py::TestSuite::test_case_1 PASSED
NuSMV/test/bug_026/test_try.py::TestSuite::test_case_2 PASSED
NuSMV/test/bug_026/test_try.py::TestSuite::test_case_3 PASSED
====================== 3 passed in 0.01 seconds ====================

In the first case there are 3 collected items, while in the second case the collected items are 12. I guess that for every test case there are 4 test collected(3 for all the test case in the file and one for the real test case to run).

If this approach is applied on a larger scale (for instance with many test files that contains many test), I observed a huge increase of memory usage.

I was wandering if there is an easy way to collect only the single test case in order to lower the number of test collected and so the memory usage?

Thanks

[RonnyPfannschmidt]

can you outline used pytest-plugins and py.test versions as well?

at first glance i fear, that we run collection multiple times on the same file, i'll try to verify this week

[fontealpina]

ok I tried both with:

platform linux2 -- Python 2.6.6, pytest-2.9.2.dev1, py-1.4.26, pluggy-0.3.1 -- /usr/bin/python

platform linux2 -- Python 2.7.6, pytest-2.9.2.dev1, py-1.4.26, pluggy-0.3.1 -- /usr/bin/python

but also with an old version:

platform linux2 -- Python 2.6.6 -- py-1.4.26 -- pytest-2.7.0.dev1 -- /usr/bin/python

[RonnyPfannschmidt]

thanks for verifying that the issue is with python 2.6 (which is EOL)

[fontealpina]

Wait a moment. I'm sorry I've updated the message because I was checking the issue in python2.7 in a wrong way. So I have updated the message. I'm obtaining the same issue both with python2.6 and python2.7. Sorry

[fontealpina]

Any news about this issue? Thanks

[RonnyPfannschmidt]

i didn't find the time to look at it again so far

[fontealpina]

Sorry, have you checked this? To me would be enough even a temporary fix to avoid it, maybe can you suggest a way to do it. Thanks

[RonnyPfannschmidt]

i still didnt manage to look into detail

my first idea idea for fixing it is to go in, and split node-id based collection, into a plugin that inhibits collection outside of the requested items

it would be a class which implements a make-item hook wrapper and discards undesirable results and it would be installed into the plugin-manager, if such tests where given as arguments

[nicoddemus]

@RonnyPfannschmidt that sounds like a good idea. Perhaps @fontealpina can implement that hook into his conftest file so we can be sure the solution will work? I would suggest implementing pytest_collect_file by looking at the path component of the test id instead if pytest_pycollect_makeitem is too complicated.