Technologicat / pyan

Static call graph generator. The official Python 3 version. Development repo.
GNU General Public License v2.0
323 stars 56 forks source link

Add support for Python 3.8 and later #51

Open Technologicat opened 3 years ago

Technologicat commented 3 years ago

We need to support ast.NamedExpr a.k.a. the walrus operator (name := value).

That's probably the only AST change that Pyan needs to be aware of. To be sure, check GTS, and mcpyrate's unparser (to which I added Python 3.8 support already).

Technologicat commented 3 years ago

Eh, there's also ast.Constant, to replace the old constant types (Num, Str, Bytes, NameConstant, Ellipsis). Python 3.6 already has ast.Constant, but doesn't use it; Python 3.8 changes the compiler to actually emit ast.Constant.

jdb78 commented 3 years ago

We currently have issues with projects that do not have an __init__.py in every directory that is a package.

Technologicat commented 3 years ago

@jdb78: good to know, thanks.

In other news, for 3.8, we should also add support for __pypackages__ (PEP 582 - Python local packages directory). The __pypackages__ directory itself needs to be omitted from the dotted name of the module when Pyan generates the fully qualified module name from a .py filename.

These should also be preferred over site-packages, but for that, maybe a simple rule to prefer earlier entries on the command line (plus instructions saying so) might be enough in practice.

Technologicat commented 3 years ago

Heads-up for 3.9: the AST format has changed again. As of February 2021, this isn't yet documented in GTS.

Particularly, at least ast.Subscript now no longer has ast.Index and ast.ExtSlice containers. Good riddance, the new design is better - but the change causes some headaches to tools that rely on the AST format. (See https://github.com/Technologicat/mcpyrate/issues/20.)

Might be that Pyan, in particular, doesn't have to care about this change - but it's something we should verify before adding a python39 label.

I don't know yet about other AST changes in 3.9 or 3.10. I'll keep an eye out for them.

Technologicat commented 2 years ago

No other AST changes in 3.9 or 3.10... except the new source location fields (end_lineno, end_col_offset), which we don't need here.