`pytest_discover_instances` causes high memory usage on moderately sized projects

[craigrosie]

I haven't seen this reported anywhere else, so I'm not sure if I'm the only one having a problem or if there's something wrong with my configuration.

I work on a project with ~400 Python files, ~150 of which are tests. If I open buffers and navigate around the codebase, neotest-python starts running neotest-python/neotest.py --pytest-collect <test file> to collect all the tests.

Even after only opening a couple of files in my codebase, I have 45 neotest-python/neotest.py --pytest-collect <test file> processes running. Sometimes these are even running for the same <test file> - as I write this I have 8 processes running for the same test file path. Additionally, these processes don't seem to exit (although if I run the command manually myself e.g. .../python .../neotest-python/neotest.py --pytest-collect /<test file>, the command completes in ~0.5s).

This means that as I continue to work on my codebase in a single Neovim session, more and more of these "pytest collect" processes get started, and never complete. Each process takes ~190mb of memory, so eventually my machine runs out of memory and I get a Your system has run out of application memory alert before everything freezes and my machine force restarts.

I absolutely love neotest and neotest-python, and the use of pytest_discover_instances to enable discovery of parametrized tests is super useful, but unfortunately it's unusable for me at the moment 😢

My neotest/neotest-python is:

require("neotest").setup({
  adapters = {
    require("neotest-python")({ dap = { justMyCode = false }, args = { "-vv" }, pytest_discover_instances = true }),
  },
  quickfix = { enabled = false },
})

I'm running Neovim v0.10.0, https://github.com/nvim-neotest/neotest-python/commit/2e83d2bc00acbcc1fd529dbf0a0e677cabfe6b50 for neotest-python and https://github.com/nvim-neotest/neotest/commit/f30bab1faef13d47f3905e065215c96a42d075ad for neotest.

I haven't had a chance to dig into the neotest-python code yet to see if there might be a simple fix, but I thought I'd raise this in case it's affecting others.

Please let me know if I can provide any more info to help debug!