Rewriting AST from python visitor

[borismarin]

Hello everyone, thanks for this great tool! I have been trying to use the python bindings to implement a visitor that rewrites the AST -- not only changing attributes, like what has been discussed in issue #702, but also appending / removing nodes.

As a concrete example, I would like to comment out TABLE statements (or even remove them), in order to use your InlineVisitor in a second pass (inlining is aborted in case 'lag' or 'table' statements are found). Directly replacing the ast.TableStatement node with ast.LineComment does not (and probably should not...) work:

     def visit_table_statement(self, node):
         this_as_string = nmodl.ast.String(nmodl.to_nmodl(node))
         node = nmodl.ast.LineComment(this_as_string)
         return node # this does not replace `node`... 

Simply pruning this node from its parent is not possible as well. Given that changing a data structure that is being iterated over seems like a bad idea, I do not expect these approaches to work. But what would be the proper way to end up with a different AST? Rebuilding a new tree by copying all nodes, only disregarding the unwanted ones, seems a possibility -- but I cannot find a way to programatically do that from python: while I can create individual nodes using their constructors, how can I join them to build the tree? the following does not work, for example:

from nmodl import ast
p = ast.Program()  # node correctly created
m = ast.Model(ast.String('foo')) # node correctly created
p.blocks.append(m) # no error
len(p.blocks) # zero!

Thanks for your help!

[1uc]

I think what's happening is that it returns a copy of the list each time you call (the property) blocks.

I tried with visit_statement_block and node.statements. What I get is that:

class StementBlockVisitor(nmodl.visitor.AstVisitor):
    def visit_statement_block(self, node):
        stmts = node.statements
        stmts.append(nmodl.ast.String("fooo"))
        node.statements = stmts
        print(type(node.statements))

has the desired effect.

[1uc]

To answer the remaining questions:

• node = String("foo") can't work because in Python this just gives String("foo") a new name: node. It doesn't assign the value String("foo") to the existing object node.
• Therefore, yes, also on the C++ side we mostly just replace entire StatementBlocks with the desired value.

[borismarin]

Thanks @1uc ! But I am still having trouble changing the statement_block node. Even using your code above, I still get a TypeError:

        stmts = node.statements                                                                                                         
        stmts.append(nmodl.dsl.ast.String("fooo"))                                                                                      
>       node.statements = stmts                                                                                                         
E       TypeError: (): incompatible function arguments. The following argument types are supported:                                     
E           1. (arg0: nmodl._nmodl.ast.StatementBlock, arg1: List[nmodl._nmodl.ast.Statement]) -> None

Note: I'm on nmodl.__version__ = '0.5' from PyPI.

[1uc]

Try updating, e.g. building master, or maybe 0.6 (but even that's old). You can also try pip installing nmodl-nightly. Since we never use the Python layer ourselves, I had to experiment to get the answer, which is why I'm sure on reasonably new versions of NMODL it works.

Also check if the difference nmodl.dsl.ast.String vs. nmodl.ast.String matters. It looks like it's complaining about the types of the arguments for:

@property
def statement(self, stmts)

It says that List[nmodl._nmodl.ast.Statement] is supported; which sounds like exactly what we'd need.

[borismarin]

Thanks @1uc.

For the record, stmts.append(dsl.ast.String("fooo")) won't work as ast.String is not a subclass of Statement (it extends Expression, though) . Wrapping it into eg a LineComment solves the problem: stmts.append(dsl.ast.LineComment(dsl.ast.String("fooo"))).

Re. the difference between types under nmodl.dsl.ast and nmodl._nmodl.ast,

import nmodl
from nmodl import dsl
assert dsl.ast.String is nmodl._nmodl.ast.String

is true for all nmodl versions tested (>0.5)