search

DFKI-NLP / DISTRE

[ACL 19] Fine-tuning Pre-Trained Transformer Language Models to Distantly Supervised Relation Extraction
https://www.aclweb.org/anthology/P19-1134
Apache License 2.0
85 stars 13 forks source link

the sentence representations? #2

Closed YaNjIeE closed 4 years ago

YaNjIeE commented 4 years ago

Hi, I wonder how you obtain the sentence representation. In your paper:

A sentence representation is obtained by feeding the token sequence xi of a sentence to the pre-trained model and using the last state of the final state representation hL as its representation si.

Is the sentence representations are the last state of '[CLS]' or something else? I am confused about that. Would you please explain that for me? Thanks a lot.

Best

ChristophAlt commented 4 years ago

Hi, your understanding is correct. The sentence representation is the last layer state of the [CLS] token.

YaNjIeE commented 4 years ago

OK, thanks a lot. Have a nice day.

charosen commented 4 years ago

@yanjieG did you reproduce the 0.422 results by training your own DISTRE model?