[BUG] False positive in status bar when running python tests

[lyz-code]

NeoVim Version

NVIM v0.8.2
Build type: Release
LuaJIT 2.1.0-beta3
Compiled by runner@fv-az190-151

Features: +acl +iconv +tui

Describe the bug When I run any python test it's marked as failed in the status line

To Reproduce Please provide a minimal init.lua to reproduce which can be run as the following:

nvim --clean -u minimal.lua

You can edit the following example file to include your adapters and other required setup.

-- ignore default config and plugins
vim.opt.runtimepath:remove(vim.fn.expand("~/.config/nvim"))
vim.opt.packpath:remove(vim.fn.expand("~/.local/share/nvim/site"))
vim.opt.termguicolors = true

-- append test directory
local test_dir = "/tmp/nvim-config-2"
vim.opt.runtimepath:append(vim.fn.expand(test_dir))
vim.opt.packpath:append(vim.fn.expand(test_dir))

-- install packer
local install_path = test_dir .. "/pack/packer/start/packer.nvim"
local install_plugins = false

if vim.fn.empty(vim.fn.glob(install_path)) > 0 then
  vim.cmd("!git clone https://github.com/wbthomason/packer.nvim " .. install_path)
  vim.cmd("packadd packer.nvim")
  install_plugins = true
end

local packer = require("packer")

packer.init({
  package_root = test_dir .. "/pack",
  compile_path = test_dir .. "/plugin/packer_compiled.lua",
})

packer.startup(function(use)
  use("wbthomason/packer.nvim")
  use  "nvim-neotest/neotest-python"

  use({
    "nvim-neotest/neotest",
    requires = {
      "vim-test/vim-test",
      "nvim-lua/plenary.nvim",
      "nvim-treesitter/nvim-treesitter",
      "antoinemadec/FixCursorHold.nvim",
    },
    config = function()
      require("neotest").setup({
        adapters = {
        require("neotest-python")({ -- https://github.com/nvim-neotest/neotest-python
          dap = { justMyCode = false },
        }),
        },
      })
    end,
  })

  if install_plugins then
    packer.sync()
  end
end)

vim.cmd([[
command! NeotestSummary lua require("neotest").summary.toggle()
command! NeotestFile lua require("neotest").run.run(vim.fn.expand("%"))
command! Neotest lua require("neotest").run.run(vim.fn.getcwd())
command! NeotestNearest lua require("neotest").run.run()
command! NeotestDebug lua require("neotest").run.run({ strategy = "dap" })
command! NeotestAttach lua require("neotest").run.attach()
command! NeotestOutput lua require("neotest").output.open()
]])

Steps to reproduce the behavior:

1. Go to any python test
2. Run :NeotestNearest
3. See error

Please provide example test files to reproduce.

Expected behavior Passing tests are marked as passed

Logs

1. Wipe the neotest.log file in stdpath("log") or stdpath("data").
2. Set log_level = vim.log.levels.DEBUG in your neotest setup config.
3. Reproduce the issue.
4. Provide the new logs.

I don't understand the instructions to wipe the log or where to find it, if you help me locate it I can give it to you

Additional context Add any other context about the problem here.

[rcarriga]

Apologies for the late response, I somehow missed this issue :sweat_smile:

Are you still experiencing this? The log file can found by running the command :echo stdpath("log") which will print a directory where there is a neotest.log file (or there will be when the log level is set to debug). Please delete the file, reproduce the issue and paste the new file that is created here

[kevinrobayna]

This also happened to me using the rspec plugin.

[kevinrobayna]

After deleting the ~/.local/state/nvim/neotest.log and closing neovim when I run the spec the state matches the results and I can't reproduce it now. If this happens to me again I'll try to get the logs and paste them here

[kevinrobayna]

I got an error on the file

❯ cat ~/.local/state/nvim/neotest.log
WARN | 2023-02-27T16:27:17Z+0000 | ...pack/packer/start/neotest/lua/neotest/lib/subprocess.lua:156 | CHILD | Error in remote call Vim:E117: Unknown function: test#test_file
stack traceback:
        [C]: in function 'test#test_file'
        ...ker/start/neotest-vim-test/lua/neotest-vim-test/init.lua:38: in function 'func'
        ...pack/packer/start/neotest/lua/neotest/lib/subprocess.lua:148: in function <...pack/packer/start/neotest/lua/neotest/lib/subprocess.lua:147>
        [C]: in function 'xpcall'
        ...pack/packer/start/neotest/lua/neotest/lib/subprocess.lua:147: in function <...pack/packer/start/neotest/lua/neotest/lib/subprocess.lua:146>

Everytime I delete the file and try again it works again

[rcarriga]

That error is because the vim-test adapter is being use (is that the correct adapter?) and vim-test is likely being lazy loaded. You should configure vim-test to be loaded with neotest-vim-test. Please open a separate issue if you have further problems, as this is unrelated to this issue.

[M-Sviridov]

Hi @rcarriga , I'm not sure whether it was best to add to this issue or create a new one.

I'm having a similar scenario as @kevinrobayna using RSpec, I am however, only using neotest. Basically when I launch nvim for the first time and go to a repo containing multiple _spec.rb files, the first file I pick to run the tests shows the correct sign. But then all subsequent tests for other _spec.rb files (in the same directory) show the wrong sign.

My neotest.log file is empty, and my config is the followoing:

 {
    "nvim-neotest/neotest",
    lazy = true,
    dependencies = {
      "nvim-lua/plenary.nvim",
      "nvim-treesitter/nvim-treesitter",
      "antoinemadec/FixCursorHold.nvim",
      "olimorris/neotest-rspec",
    },
    config = function()
      require("neotest").setup {
        adapters = {
          require "neotest-rspec",
        },
        icons = {
          passed = "",
          running = "",
          failed = "",
          unknown = "",
        },
        quickfix = { open = false },
        status = { virtual_text = false },
        output = { open_on_run = false },
        lazy = true,
      }
    end,
  },

See the screenshots attached.

Correct:

Wrong:

Cheers and thank you!

[neeerp]

I had just set up neotest with the neotest-python adapter for the first time and was running into this issue. Here's a log file (I believe I ran all the tests in the file a couple of times, and then ran some individual tests afterwards). In all cases, the tests pass but they're marked by neotest as failing.

neotest.log

[neeerp]

I noticed that this doesn't happen on other projects. The test suite of https://github.com/pytest-dev/pytest (which unsurprisingly also uses pytest) marks passing tests as passing.

The screenshot in my previous comment is of https://github.com/python-poetry/poetry, where passing tests are incorrectly marked as failing.

[neeerp]

It looks like this might be related to parallel tests runs (at least in my case):

Removing the -n option here that tells pytest to use that many subprocesses makes the tests get marked correctly.

[neeerp]

Created this repo with a minimal repro for the issue I ran into with pytest: https://github.com/neeerp/parallel-neotest-bug-repro

[eerison]

Is there any way to disable this icon? just until solve this issue?