search

autoliuweijie / K-BERT

Source code of K-BERT (AAAI2020)
https://ojs.aaai.org//index.php/AAAI/article/view/5681
949 stars 212 forks source link

extend to other pre-trained LMs #67

Open lytum opened 3 years ago

lytum commented 3 years ago

Hi,

thanks for sharing the excellent work. May I ask if this knowledge-graph based model can be extended into other pre-trained models, like RoBERTa or GPT2? My understanding is Yes. Could you give me more explanation about that? Thanks a lot!!

autoliuweijie commented 3 years ago

Hi,

thanks for sharing the excellent work. May I ask if this knowledge-graph based model can be extended into other pre-trained models, like RoBERTa or GPT2? My understanding is Yes. Could you give me more explanation about that? Thanks a lot!!

Yes, the model structures of RobERTa and BERT are the same.