Bug report for the AMR model

[zhijing-jin]

Hi thanks for making this wonderful package. Very user-friendly and easy to use!

I am using the AMR parsing function. And this post is to report that there are some failure cases, where the generated AMR is quite extreme:

sent = 'There are 300 masks / 10 masks/baggy = 30 baggies face masks.'
import elit
parser = elit.load(elit.pretrained.amr.AMR3_BART_LARGE_EN)
amr = parser(sent)
print(amr)

The generated AMR has thousands of variables, up to "x2000". And it seems to enter into a recurrent loop as follows:

(z1 / equal-01
    :ARG1 (z2 / product-of
              :op1 300
              :op2 10
              :op3 (z3 / product-of
                       :op1 (z4 / product-of
                                :op1 (z5 / product-of
                                         :op1 (z6 / product-of
                                                  :op1 (z7 / product-of
                                                           :op1 (z8 / product-of
                                                                    :op1 (z9 / product-of
                                                                             :op1 (z10 / product-of
                                                                                       :op1 (z11 / product-of
                                                                                                 :op1 (z12 / product-of
                                                                                                           :op1 (z13 / product-of

It's just a bug for your record.

[hankcs]

Hi @zhijing-jin , thanks for reporting this bug. It's probably due to the famous "text degeneration" problem of many seq2seq models. After https://github.com/emorynlp/elit/commit/0dad49e587ce415e867f1c6aaf39a5261ebde9c4 is released, you can set beam_size=4, no_repeat_ngram_size=4 to mitigate it (probably tune no_repeat_ngram_size on some dev set).

parser(sent, beam_size=4, no_repeat_ngram_size=4)

(z1 / equal-01
    :ARG1 (z2 / product-of
              :op1 300
              :op2 10
              :op3 (z3 / baggy))
    :ARG2-of (z4 / equal-01
                 :ARG1 (z5 / mask
                           :mod (z6 / face)
                           :mod (z7 / baggie))))

Though the result is still not perfect, it looks better now.

[zhijing-jin]

Great, thank you! This is really helpful :)!!