search

bjascob / amrlib

A python library that makes AMR parsing, generation and visualization simple.
MIT License
219 stars 34 forks source link

Spring parser returning nothing for certain strings #45

Closed JosephGatto closed 2 years ago

JosephGatto commented 2 years ago

Wanted to share some weird behavior I found

For the SPRING parser, I get no graph returned for the following strings

'advertisement To estimate the risk of death from heart disease most doctors use a calculator endorsed by the American Heart Association and the American College of Cardiology'

To muddy the waters further In 2016 a systematic review revealed no association with LDL cholesterol and heart disease in those aged over sixty and an inverse association with all cause mortality in other words the higher your cholesterol in this age group the longer you would live 15

Your body needs cholesterol to Promote brain health

Not sure why! Thought it might be of interest.

bjascob commented 2 years ago

In a very small number of cases, either of the transformers based parse models (parse_xfm and parse_spring) may return an output string that the de-serializer can't figure out how to convert back into a well formed graph. In this case you just get an empty return for the graph. I believe the parse_xfm code is a little better handling errors and you're more likely a good return.

Failed parses don't happen very often. During training/test with parse_xfm I would sometimes see 1 out of 1722 graphs from the AMR3 Dev set fail to return using num_beams=1. With num_beams=4 (the default in Inference.py), I never saw any of the Test or Dev set graphs fail to return. However, if you run enough sentences through it, you'll probably run into a few eventually.

JosephGatto commented 2 years ago

Thank you as always for the quick feedback! This was helpful.