search

atcbosselut / comet-commonsense

Code for ACL 2019 Paper: "COMET: Commonsense Transformers for Automatic Knowledge Graph Construction" https://arxiv.org/abs/1906.05317
Apache License 2.0
671 stars 124 forks source link

Why do not excludes "none" answers in ATOMIC datasets? #30

Closed yongho94 closed 3 years ago

yongho94 commented 3 years ago

Hello, Sir.

I have something to ask you about this code.

I wonder a reason of why you do not exclude 'none' target answer.

Your code seems to include such data samples like "PersonX loses personX's sight xNeed none " as a training data or test data.

I think learning to generate 'none' dose not seems to meaningful.

And I found same thing in COMET-ATOMIC 2020.

Is there any specific reasons?

Thanks.

Kelaxon commented 3 years ago

@yongho94 I'm also confused about it. Did you figure it out?

atcbosselut commented 3 years ago

Commonsense knowledge graphs are incomplete (i.e., missing commonsense facts). There's a semantic difference between an annotation that says there is NO effect, intent, reaction, etc. to an event, and the lack of any annotation. One teaches the model there is NO precondition/postcondition. One teaches the model nothing.

When we use the model on new events (e.g., see the generated event graphs at the end of the paper), it can be useful to generate "none" as an effect, reaction, etc. if there are no pre-/post-conditions associated with that event.