Fix plugin for long file paths with projects using mypy pretty formatting

metropetrol commented 10 months ago

What is this change

Use mypy's --no-pretty argument for all runs to ensure that the output is parsable.

Why is this change needed

I work on large python projects where file paths can be really long and this plugin doesn't currently work for files with long paths.
mypy will use its "pretty" formatting by default and for very long file paths the output is formatted differently. This plugin's parser does not work for such output. Using the --no-pretty works for the parser and appears to be an easier change than updating the parser to support this extra output format. ^ see latest comment for more accurate description

Logs reproducing the issue and validating the fix

I pulled these logs from the plugin, they show the issue and validate that my change fixes the issue

### Diagnostic logs for long file path without this fix, doesn't work

2023-11-11 15:12:12,387 UTC - DEBUG - pylsp_mypy.plugin - report:
Incompatible types in assignment (expression has type "str", variable has type
"int")  [assignment]
    num: int = "str"
               ^~~~~

2023-11-11 15:09:43,643 UTC - DEBUG - pylsp_mypy.plugin - errors:

2023-11-11 15:12:12,387 UTC - DEBUG - pylsp_mypy.plugin - parsing: line = 'src/site_connect_grantor_service/dependencies/dynamo/grants_table.py:32:12:32:16: error:'
2023-11-11 15:12:12,387 UTC - DEBUG - pylsp_mypy.plugin - parsing: line = 'Incompatible types in assignment (expression has type "str", variable has type'
2023-11-11 15:12:12,387 UTC - DEBUG - pylsp_mypy.plugin - parsing: line = '"int")  [assignment]'
2023-11-11 15:12:12,387 UTC - DEBUG - pylsp_mypy.plugin - parsing: line = '    num: int = "str"'
2023-11-11 15:12:12,387 UTC - DEBUG - pylsp_mypy.plugin - parsing: line = '               ^~~~~'
2023-11-11 15:09:43,693 UTC - INFO - pylsp_mypy.plugin - pylsp-mypy len(diagnostics) = 0

### Diagnostic logs for short file path without this fix, does work

setup.py:5:12:5:16: error: Incompatible types in assignment (expression has
type "str", variable has type "int")  [assignment]
    num: int = "str"
               ^~~~~

2023-11-11 15:15:30,115 UTC - DEBUG - pylsp_mypy.plugin - errors:

2023-11-11 15:15:30,115 UTC - DEBUG - pylsp_mypy.plugin - parsing: line = 'setup.py:5:12:5:16: error: Incompatible types in assignment (expression has'
2023-11-11 15:15:30,116 UTC - DEBUG - pylsp_mypy.plugin - parsing: line = 'type "str", variable has type "int")  [assignment]'
2023-11-11 15:15:30,116 UTC - DEBUG - pylsp_mypy.plugin - parsing: line = '    num: int = "str"'
2023-11-11 15:15:30,116 UTC - DEBUG - pylsp_mypy.plugin - parsing: line = '               ^~~~~'
2023-11-11 15:15:30,116 UTC - INFO - pylsp_mypy.plugin - pylsp-mypy len(diagnostics) = 1

### Diagnostic logs for long file path with this fix, does work

src/site_connect_grantor_service/dependencies/dynamo/grants_table.py:32:12:32:16: error: Incompatible types in assignment (expression has type "str", variable has type "int")  [assignment]

2023-11-11 18:19:04,614 UTC - DEBUG - pylsp_mypy.plugin - errors:

2023-11-11 18:19:04,615 UTC - DEBUG - pylsp_mypy.plugin - parsing: line = 'src/site_connect_grantor_service/dependencies/dynamo/grants_table.py:32:12:32:16: error: Incompatible types in assignment (expression has type "str", variable has type "int")  [assignment]'
2023-11-11 18:19:04,615 UTC - INFO - pylsp_mypy.plugin - pylsp-mypy len(diagnostics) = 1

metropetrol commented 10 months ago

I just realized that this change breaks some of the tests, whoops

Need to fix that before review

metropetrol commented 10 months ago

Alright I fixed the test and added one that confirms the fix.

I discovered more info on what the bug actually is. It appears to be isolated to projects that explicitly configure the mypy pretty option to True. The plugin will read that configuration and use that instead of the default no-pretty formatting which breaks diagnostics for some cases like long file paths.

(I erroneously stated that mypy uses pretty formatting by default in the original description, I fixed that)

There doesn't seem to be any harm in adding this since the default is --no-pretty anyways. I think it is worth adding to make the plugin behave as consistently as possible. Hopefully this will save someone from discovering the same issue in the future. My project runs mypy and outputs the errors on build and pretty formatting is nice for that. I ideally would like to keep the pretty setting there if possible.

metropetrol commented 10 months ago

I discovered a way to fix this for my local setup by using the plugin's overrides configuration in my editor config:

overrides = [true, "--no-pretty"]

This is a fairly niche configuration issue so I'm open to this being closed. It may be useful for others though.

python-lsp / pylsp-mypy