[mycpp] annotate CFG with function calls

[melvinw]

This commit adds the first concrete Fact and uses it to annotate statements that make function calls. You can feed this in to _bin/datalog/call-graph and see a list of statements that might lead to the GC being invoked. This output still doesn't impact the generated C++, though.

[andychu]

Thinking about this a bit more, I think ideally we want a data flow structure like

    MyPy AST for Typed Python 
--> mycpp AST resembling C++ 
--> forward decls    # this may turn into proper modules eventually, instead of 1 huge translation unit
--> DECL - prototypes / class definitions with members
--> control flow     # I suppose this goes here because it only affects IMPL?   Could be up for grabs
--> IMPL   # write function bodies

[andychu]

And we would work with the separate mycpp.asdl for ALL the passes

But this is too big a change to do all at once.

---

So as an intermediate step, we could create a new first phase -- maybe the heuristic mycpp disambiguation pass that fills in what's missing in Python

• obj->field vs. module::Func() or module.Class() -- using the imported name heuristic
• member vars -- these are not listed explicitly in Python -- there is a heuristic
• virtual -- this does not appear in the source
• casting -- there are some special considerations here -- we are using the UP_foo convention
• I just added a ctx_Foo convention for rooting, since context managers only go on the stack

So instead of creating a whole new tree right now, this first phase could just output a bunch of dicts keyed by nodes

e.g. for every MemberExpr, we can create a dict in the first pass that says if it's obj->field or module::Func

And then we can use that in all passes.

Likewise for the local_var_list -- that is used for "hoisting" declarations to the top of every function, before we see them.

---

And then gradually we can move virtual there, and ctx_member_vars, etc.

We could even call this the "HeuristicPass" or something

So it would be something like

member_expr_lookup = ...
local_vars = ...
p1 = heuristic_pass.Build(member_expr_lookup, local_vars, ...)

after constructing this: DECL pass, control flow pass, IMPL pass all walk the same tree structure, and consume the dicts

So right now it would output a bunch of dicts

But later it could construct a real IR like mycpp.asdl

[andychu]

So basically I think this is a good way to gradually move stuff out of cppgen_pass

The problem is we want to use self.imported_names to check MemberExpr in both control flow and in IMPL

But we don't want to duplicate that logic. Instead we can move it to the EARLY heuristic pass, and use that info in as many subsequent stages as we want

So I think it should be a relatively flexible way of organizing the code, and lead to many quick refactorings

---

Also I like the idea of moving all the heuristics to ONE place.

And then the rest of the code eventually should some a TIGHT IR mycpp.asdl that has no "illegal states representable"

Right now we have many cases where slightly wrong code will generate invalid C++ -- there are examples on Zulip

[andychu]

Hm I tried to delete some of the "pass through" methods from const_pass.py and it didn't work

I think the problem is that we are not using that LATER API traverser.py

I probably copied from some earlier API, ugh

So yeah maybe we don't worry about it now, although this causes a bunch of boilerplate I would like to avoid

Maybe the heuristic_pass can be the same thing as the const_pass ... logically that doesn't seem too bad -- all we are doing is building a dict const_lookup

And then we can add a couple other dicts there too

[melvinw]

closing this while we figure out the IR stuff

[melvinw]

i guess there's no reason we can't just tack the IR stuff onto this change. reopening.

[melvinw]

Nitpick: I feel like "ast_pass" is a bit of generic name ... aren't they all AST passes sorta?

Ah lol good point. I'll rename this to ir_pass.

[andychu]

OK great, thanks for doing this!

I realize it was a lot of yak shaving, but I think this is the right thing for the future, and also the right time to do it, because everything works now :)

[melvinw]

No sweat!