LittlePea13
commented
2 years ago Hi there, that's quite funny. The model seems to suffer from hallucination, which is expected since the pretraining phase biases it to the Wikipedia domain, and mainly the starting text from a Wikipedia page, which may lead to this weird behavior.
While one can use REBEL as it is, I would suggest finetuning for an specific use or domain, which should alleviate this behavior. You could also look at constrained decoding when performing inference, restricting the generation of subjects and objects to the text appearing in the input.
Using coreference resolution also seems like a cool way of working around it, thanks for sharing your insights 😊
I'm testing the library, and I'm getting some weird results:
for the following input:
However, for much of the later part of his career, he worked on two ultimately unsuccessful endeavors. First, despite his great contributions to quantum mechanics, he opposed what it evolved into, objecting that nature "does not play dice". Second, he attempted to devise a unified field theory by generalizing his geometric theory of gravitation to include electromagnetism. As a result, he became increasingly isolated from the mainstream of modern physics.I get the following results:
{'relation': 'member of', 'head_span': 'John Wayne', 'tail_span': 'The Doors'} {'relation': 'member of', 'head_span': 'John Wayne', 'tail_span': 'The Rolling Stones'} {'relation': 'has part', 'head_span': 'The Doors', 'tail_span': 'John Wayne'} {'relation': 'has part', 'head_span': 'The Doors', 'tail_span': 'John Wayne'} {'relation': 'has part', 'head_span': 'The Doors', 'tail_span': 'John Wayne'} {'relation': 'facet of', 'head_span': 'opposed what it evolved into', 'tail_span': 'quantum mechanics'} {'relation': 'has part', 'head_span': 'field theory', 'tail_span': 'electromagnetism'} {'relation': 'part of', 'head_span': 'electromagnetism', 'tail_span': 'field theory'} {'relation': 'different from', 'head_span': 'modern physics', 'tail_span': 'mainstream'} {'relation': 'different from', 'head_span': 'mainstream', 'tail_span': 'modern physics'}
It seems that the model has issues with pronouns... When I run coreference resolution before, John Wayne and the Doors don't appear :)