Specialized version of Python AST?

[AndrewTolmach]

Life might be considerably easier if instead of using the standard Python Ast module directly, we built custom Ast dataclasses covering just the subset covered by the compiler, and translated to this as the very first thing (with the translator provided to the students, of course). Some of the potential advantages:

• No need to use or explain the weird AST forms that arise because the Python Ast covers more general forms than we use, e.g. in Compare, Subscript, BoolOp vs BinOp, FunctionDef, etc.
• Can define str and repr functions naturally as part of dataclass definition, avoiding the current hackery of adding them in utils.py --- which seems to break tools like mypy.
• Can do initial simplifying rewrites "under the hood" before the student ever sees the program. Examples: converting UnaryOp(Usub(),Constant(n)) to Constant(-n) (in order to make min_int representable) and simplifying function parameters (as is currently done in type_check_Lfun).

[jsiek]

I sympathize with this idea. The choice to stick with the standard Python ast module was an early decision, and it has caused more trouble than I anticipated. However, it's too late to make a change this big to the first edition of the python book. (We're currently in the later stages of copy editing.) Let's keep this in mind for further down the road.

[jnear]

btw I think this is a good idea and this is how I'm teaching it in my class this semester. I use abstract syntax like this (for chapter 2):

op ::= "add"
Expr ::= Var(x) | Constant(n) | Prim(op, [Expr])
Stmt ::= Assign(x, Expr) | Print(Expr)
LVar ::= Program([Stmt])

Last year, I found that students had lots of trouble with the weird AST forms. This semester (with the simplified abstract syntax) is going much better.

My AST is here and my translator is here.