lark-parser / lark

Lark is a parsing toolkit for Python, built with a focus on ergonomics, performance and modularity.
MIT License
4.64k stars 397 forks source link

Can not chain or merge two transformers #1359

Closed reSHARMA closed 8 months ago

reSHARMA commented 8 months ago

What is your question?

I've got two transformer, the first one does nothing, it only returns the nodes unchanged. After running the first transformer neither the output of .pretty() is same nor I'm able to run another transformer on the resulting tree. I also tried merge_transformers.

If you're having trouble with your code or grammar

The grammar is very simple but let me know if you need the whole thing, I'm only doing:

tree1 = pass1.transform(tree)
tree2 = pass2.transform(tree1) 

Explain what you're trying to do, and what is obstructing your progress.

I want to write simple passes as transformer in my DSL compiler. I want to chain multiple transformers into the tree which is generated after the first parsing. Is it a good use case for lark?

MegaIng commented 8 months ago

Can you show the transformer? An empty subclass of Transformer does nothing, and this will work as you expect. So I would imagine your pass1 does something else.

reSHARMA commented 8 months ago

Pass1:

@v_args(inline=True)
class pass1(Transformer):
    @v_args(inline=False)
    def assign(self, data):
        return data

    @v_args(inline=False)
    def instruction(self, data):
        return data

also pasting more snippets

MegaIng commented 8 months ago

Yeah, that is not a NOP. data is not a tree instance, but a list, so you are replacing some Tree instances with list instances. Make sure to always return a Tree instance from your Transformer methods, and then you will be able to chain Transformers however you want.

reSHARMA commented 8 months ago

Many thanks, so returning Tree(data) will work?

MegaIng commented 8 months ago

No, you need to construct a correct Tree instances, i.e. you need to provide a data attribute. Tree('assign', data) for example would work to make your assign method a complete NOP.

erezsh commented 8 months ago

@reSHARMA You probably want to use v_args(tree=True)

Like this:

@v_args(tree=True)
class pass1(Transformer):
    def assign(self, tree):
        return tree

This is essentially a nop, and is equivalent to

class pass1(Transformer):
    pass

As for merge_transformers, it's meant to apply transformers to a tree that contains several "namespaces". If you just want to chain transformer t1 and t2, you can do so with t1 * t2.

reSHARMA commented 8 months ago

Million thanks both of you, it works like charm now!!