search

DamnWidget / anaconda

Anaconda turns your Sublime Text 3 in a full featured Python development IDE including autocompletion, code linting, IDE features, autopep8 formating, McCabe complexity checker Vagrant and Docker support for Sublime Text 3 using Jedi, PyFlakes, pep8, MyPy, PyLint, pep257 and McCabe that will never freeze your Sublime Text 3
http://damnwidget.github.io/anaconda/
GNU General Public License v3.0
2.21k stars 260 forks source link

Linting outline in a wrong place for f-strings #631

Open riazanovskiy opened 7 years ago

riazanovskiy commented 7 years ago

Expected Behaviour

If a formatted string contains undefined variable, that string or that very variable should be outlined.

Actual Behaviour

The first line of a file is outlined if that file contains any incorrect f-string. The status bar shows correct message in that case ("Undefined name ...")

Steps to Reproduce

Create following python file

# 
print(f"{x}")

ST3, Anaconda and OS versions

ST version 3126, Anaconda 2.1.17, python 3.6.0, 64 bit Arch Linux Same on Windows 8.1

ST3 Console Logs

STlogs.txt

Anaconda's JsonServer Logs

anaconda_jsonserver.txt

DamnWidget commented 7 years ago

Looks like this comes directly from PyFlakes And it is already reported: https://bugs.launchpad.net/pyflakes/+bug/1666811

DamnWidget commented 7 years ago

I have been investigating this much deeper and seems like the AST coming from the compiled code object that comes directly from the Python compiler itself sets the line number as 1 for that specific node into the tree what seems to be wrong but I do not really know how the f-string PEP is internally implemented

DamnWidget commented 7 years ago

And digging a bit more into the CPython internals I found that the node lineno is set as it is extracted with the LINENO macro when the f-string is being parsed but for some reason it does not provides correct line numbers. I headed to the Python issue tracker to fill a new bug about it but seems like there is already one going on.

Do not expect this to be fixed any time soon.

DamnWidget commented 7 years ago

Just an update to understand why this is happening (just a reminder for myself really). The literal formatted string is just syntactic sugar, the Python interpreter is parsing the code and creating tokens, then those tokens are used by the lexical analyzer to create a concrete syntax tree (CST) that is then converted into a much more useful abstract syntax tree (AST) that is what PyFlakes and other analysis tools finally use.

The f-string implementation is a bit lazy or sloppy if you prefer and instead of create a new token type like FSTRING and then properly analyze the expression to split it into more tokens it just created a new token of type STRING like in our example:

STRING "f'{x}'"

This is later parsed by the fstring_find_expr function from Python/ast.c that after perform some checks like the literal contains { characters (for the wildcards) it just generates a new string on the fly using the fstring_compile_expr function (also defined in Python/ast.c) thar returns an expr_ty taht is then formatted.

The problem is that new generated expression loses any context information so it defines by default line and column numbers (being both 1) for the enclosed variable names (that are created directly as an AST node by PyParser_ASTFromString in Python/pythonrun.c).

I would like to contribute with a fix for this directly into the CPython project but I think the implementation is not the best that Python as seen so I do not feel like doing it really.

ScriptProdigy commented 4 years ago

My 2 cents; this bug is incredibly annoying.

1 f string in a file suddenly makes all other anaconda errors, warnings, etc all disappear from view.

Resolution after several years of this known issue would be outstanding <3