lark-parser / lark

Lark is a parsing toolkit for Python, built with a focus on ergonomics, performance and modularity.
MIT License
4.8k stars 409 forks source link

Evaluate list of trees #172

Closed azarezade closed 6 years ago

azarezade commented 6 years ago

How can I transform tree, when using ambiguity="explicit" in the parser.

    parser = Lark(grammar, start="start", parser="earley", ambiguity="explicit")
    query = u"bla bla bla..."
    trees = parser.parse(query)
    print(Evaluate().transform(trees))

I have also tried to iterate over subtrees then Evaluate() them, but it also returns error!

for tree in trees.iter_subtrees():
    print(Evaluate().transform(tree))

It return TypeError: 'Tree' object is not subscriptable.

erezsh commented 6 years ago

It looks like the issue is with your implementation of the Evaluate class.

azarezade commented 6 years ago

@erezsh I don't think so, since my Evaluate class works perfectly when the parser and tree are:

    parser = Lark(grammar, start="start", parser="earley")
    tree = parser.parse(query)

but when I use:

    parser = Lark(grammar, start="start", parser="earley", ambiguity="explicit")
    trees = parser.parse(query)

which returns a list of trees, then the Evaluate dos not work. Should I use Evaluate on the trees or iterate over subtrees of trees then Evaluate? I tried both and it doesn't work.

erezsh commented 6 years ago

@azarezade

The parser never returns a list of trees. It always returns a single Tree instance. When there is an ambiguity, it's signified by an _ambig node. i.e. tree.data == '_ambig'

If you fail to account for this situation, it might cause the error you reported.

azarezade commented 6 years ago

Thanks @erezsh.

Yes, I know that! This is why I used iter_subtrees() method to iterate over the different subtrees that resulted because of the ambiguity.

Maybe my question is not clear enough. I am seeking for a way to choose the best parse tree (subtree) when there is an ambiguity. For this means, first I need a way to Evaluate all subtrees, then I choose the best one based on the evaluation result.

How can I Evaluate each subtree when there is ambiguity?

erezsh commented 6 years ago

@azarezade

I think we're going in circles a bit. Perhaps you can write the full exception you're getting?

azarezade commented 6 years ago

When I run:

parser = Lark(grammar, start="start", parser="earley", ambiguity="explicit")
query = u"wake me up tomorrow at 8 am"
tree = parser.parse(query)
for t in tree.iter_subtrees():
    print(EvalTime().transform(t))

it return this exception:

/Library/Frameworks/Python.framework/Versions/3.6/bin/python3 "/Users/ali/Google Drive/Simulations/nlu/parser_lark.py"
Traceback (most recent call last):
  File "/Users/ali/Google Drive/Simulations/nlu/parser_lark.py", line 563, in <module>
    print(EvalTime().transform(tree))
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/lark/tree.py", line 121, in transform
    return f(items)
  File "/Users/ali/Google Drive/Simulations/nlu/parser_lark.py", line 283, in time
    hour = args[0]['hour']
TypeError: 'Tree' object is not subscriptable

But when I run the following code it works without any problem. Although, the results is not desired for me, sine I need to check all ambig subtrees.

parser = Lark(grammar, start="start", parser="earley")
query = u"wake me up tomorrow at 8 am"
tree = parser.parse(query)
for t in tree.iter_subtrees():
    print(EvalTime().transform(t))
erezsh commented 6 years ago

So, the reason you get this error is because in

hour = args[0]['hour']

The value of args[0] is a Tree instance, probably an _ambig node. You'll have to handle it yourself.

How can I Evaluate each subtree when there is ambiguity?

That is how.

erezsh commented 6 years ago

P.S. You can do

print( args[0].pretty() )

To see what you're dealing with. You can also handle the _ambig nodes even before you transform, by doing:

for ambig_node in tree.find_data('_ambig'):
    do_something()
azarezade commented 6 years ago

Thanks!