About entities or relation encoding.

Neph0s / LMKE

Code for the paper 'Language Models as Knowledge Embeddings'

53 stars 7 forks source link

Closed qiucy23 closed 1 year ago

qiucy23 commented 1 year ago

In Dataloader.py , it means h/r/t is encoded by [CLS] token, but same token equals same representation, so i'm so confuse about it.

Neph0s commented 1 year ago

Please refer to model.py and search the key word 'self.mask_embeddings'.

The embedding of [CLS] token will be replaced by different embeddings corresponding to h/r/t respectively.