Unable to view python output with "Trouble quickfix"

[aabbccddeeffggj]

Did you check docs and existing issues?

• [X] I have read all the trouble.nvim docs
• [X] I have searched the existing issues of trouble.nvim
• [X] I have searched the existing issues of plugins related to this issue

Neovim version (nvim -v)

0.9.4

Operating system/version

Arch Linux

Describe the bug

I cannot see the quickfix output with "Trouble quickfix" when compiling a python file, when opening the quickfix window I can't see it there.

Steps To Reproduce

1. Compile python file.
2. :Trouble quickfix
3. No quickfix output displayed there.

Expected Behavior

The quickfix output display in Trouble's window.

Repro

-- DO NOT change the paths and don't remove the colorscheme
local root = vim.fn.fnamemodify("./.repro", ":p")

-- set stdpaths to use .repro
for _, name in ipairs({ "config", "data", "state", "cache" }) do
  vim.env[("XDG_%s_HOME"):format(name:upper())] = root .. "/" .. name
end

-- bootstrap lazy
local lazypath = root .. "/plugins/lazy.nvim"
if not vim.loop.fs_stat(lazypath) then
  vim.fn.system({ "git", "clone", "--filter=blob:none", "https://github.com/folke/lazy.nvim.git", lazypath, })
end
vim.opt.runtimepath:prepend(lazypath)

-- install plugins
local plugins = {
  "folke/tokyonight.nvim",
  "folke/trouble.nvim",
  -- add any other plugins here
}
require("lazy").setup(plugins, {
  root = root .. "/plugins",
})

vim.cmd.colorscheme("tokyonight")
-- add anything else here

[aabbccddeeffggj]

Just now I was testing with c files, which where working with Trouble, but now suddenly they stop working too. Tested with haskell and this one still working at least. That's really strange.

[tummetott]

what happens if you open the built-in quickfix list with :copen. Is the quickfix list full with entries?

[aabbccddeeffggj]

what happens if you open the built-in quickfix list with :copen. Is the quickfix list full with entries?

Yes, I forgot to mention that. All the compilers displays in the built-in quickfix, but in trouble the python compiler didn't work. About the c compiler above (gcc), actually, this one is still working, I just missplaced some characters in the input.

What I realized is: both the compilers of c (gcc) and haskell (ghc) output the errors like "error: ", with the "error:" highlighted; python just output a single error without any highlight, and without the "error: " structure. Maybe Trouble uses patterns to display the errors and doesn't have one for the default python compiler? (though it seems unlikely, since it's one of the most used compilers now)

I'm suspecting I need to insert the right argument in the python command; will try it now (edit: that's not it).

[tummetott]

could you maybe make a screenshot of the qflist populated with python errors?

[aabbccddeeffggj]

Sure, thanks for helping me: As you can see, it stops right in the first error, so it gets populated only by a single error. I'm not sure if you meant this, or something from vim.fn.getqflist(), I'm still trying to understand this part about quickfix context.

[tummetott]

What command did you use to populate the native qflist with this error? Did you read the error from a file, do you use :make or another method

[aabbccddeeffggj]

I first used set makeprg=python, then used make 01.py. When using the python compiler I noticed it only displays the first error it finds without any highlighting, while the other compilers (gcc, g++, ghc) display all errors with their respective highlights; I think it might have something to do with it.

[tummetott]

https://stackoverflow.com/questions/61808721/why-does-python-print-only-one-error-message

[aabbccddeeffggj]

https://stackoverflow.com/questions/61808721/why-does-python-print-only-one-error-message

Understood, thanks. I wonder how I can reddirect this raw output to Trouble, as I would like to have all the compilers under the same thing. I think I can make the output to a file and someway reddirect it to some mode in trouble, but not sure yet how.

[tummetott]

The challenge at hand arises from Python's outputting of errors in an incompatible format. Neovim attempts to parse this output using errorformat but struggles to recognize the pattern. Consequently, the generated quickfix entries are incorrect. When inspecting the quickfix list with vim.fn.getqflist({ all = true }), it becomes apparent that Neovim places every line of the error message in the text field, failing to recognize essential details such as the error's line number and type.

{
    bufnr = 0,
    col = 0,
    end_col = 0,
    end_lnum = 0,
    lnum = 0,
    module = "",
    nr = -1,
    pattern = "",
    text = '  File "/Users/jonas/test/test.py", line 6',
    type = "",
    valid = 0,
    vcol = 0
},
{
    bufnr = 0,
    col = 0,
    end_col = 0,
    end_lnum = 0,
    lnum = 0,
    module = "",
    nr = -1,
    pattern = "",
    text = "    for i in range(4-):",
    type = "",
    valid = 0,
    vcol = 0
},
{
...
}

The critical aspect here is the valid field, consistently set to 0. When using :Trouble quickfix, Trouble reads all quickfix entries, but if none are valid, it skips them. Hence, when viewing Trouble, no entries are displayed, and opening the native quickfix yields incorrectly formatted entries.

The Fundamental Solution: The key objective is to have our makeprg output errors in a format that Vim comprehends. Unfortunately, configuring the python3 command for error formatting is not feasible. However, linters such as pylint or flake8 can be utilized:

pylint --reports=no --output-format=text --msg-template="{path}:{line}:{column}:{C}:[{msg}]"

This command instructs pylint to print errors in the format: {file_path}:{line_of_error}:{column_of_error}:{error_type}:[{error_message}] (see documentation).

The next step involves configuring Neovim to interpret errors in the specified format. Set the errorformat:

:set errorformat=%f:%l:%c:%t:[%m]

Since each makeprg behaves differently, these settings should apply exclusively to Python files. Add the following lines to after/ftplugin/python:

vim.opt.makeprg = 'pylint --reports=no --output-format=text --msg-template="{path}:{line}:{column}:{C}:[{msg}]" %'
vim.opt.errorformat = '%f:%l:%c:%t:[%m]'

With these configurations, running :make in Neovim for Python files should yield correctly formatted quickfix entries, compatible with tools like Trouble.

BUT:

Linters like pylint or flake8 show (like the python interpreter) only one error at a time (but several warnings, hints). If you just want to investigate diagnostics in a trouble window, I suggest to set up a language server for python (like pyright) and then you can run :Trouble workspace_diagnostics which shows you all errors, warnings, hints, etc.

[tummetott]

In my opinion, you should rather use :make for compiled languages. The idea behind make is to catch compilation errors. It always makes more sense to setup linters with your LSP instead of makeprg. This means you're not limited to viewing results only in a quickfix list; you can also display them in your sign column or as virtual text.

[aabbccddeeffggj]

I see, thanks a lot, that solved the problem. I already got the pyright LSP set, but as you have have mentioned, it seems like the quickfix itself for the code interpreter can be set aside since the language server already do the job for languages like python.