[Parser] Expression tree is not properly generated in `FormattedValue`

[akshanshbhatt]

FormattedValue field of a fstring does not generate a proper expression tree for the expression inside it.

Example -

$ cat examples/expr2.py
f'''
a is {a}
b is {b}
sum is {a + b}
'''
$ lpyast examples/expr2.py
(Module [(Expr (JoinedStr [(ConstantStr "
a is " ()) (FormattedValue (Name a Load) -1 ()) (ConstantStr "
b is " ()) (FormattedValue (Name b Load) -1 ()) (ConstantStr "
sum is " ()) (FormattedValue (BinOp (Name a Load) Add (Name b Load)) -1 ()) (ConstantStr "
" ())]))] [])
$ lpynewpar examples/expr2.py
(Module [(Expr (JoinedStr [(ConstantStr "
a is " ()) (FormattedValue (Name a Load) -1 ()) (ConstantStr "
b is " ()) (FormattedValue (Name b Load) -1 ()) (ConstantStr "
sum is " ()) (FormattedValue (Name a + b Load) -1 ()) (ConstantStr "
" ())]))] [])

[certik]

Currently a + b is a string, and we put it into a Name. Rather, we will need to parse this string properly, using the parser itself, to obtain (BinOp (Name a Load) Add (Name b Load)). It would be nice to reuse the parser that we have. This expression I assume can be arbitrarily complicated, such as a+sin(x)+f*10**y - g(a, b, c, d = x). I don't know if our parser has some global state or not. If not, then it should be possible to instantiate it and call it on this string to obtain AST, while parsing the main file. If there is a global state, then probably the best way is to store a "flag" that the generated AST will have to be visited after wards. Then we write an AST visitor, that goes into each FormattedValue and calls the parser on the string to get BinOp. This walker will slow things down, so we only want to call it if the "flag" is on, which will only happen if there is an expression like "a+b". Most use cases are just simple variables I think, and those we can parse already.