search

allenai / kb

KnowBert -- Knowledge Enhanced Contextual Word Representations
Apache License 2.0
370 stars 50 forks source link

How to inject knowledge into other pre-trained models #37

Open Luoyumeng opened 2 years ago

Luoyumeng commented 2 years ago

Hello,I am new at this and I want to know how to inject knowledge into other pre-trained models. Can you give me some pointers? If I can get your help, I will be very grateful.

gpiat commented 2 years ago

In theory, you should be able to specify a different model than BERT-base-uncased (as long as it's a BERT-type model supported by pytorch-pretrained-bert) in the JSONNET files in the training_config subdirectory of this project. You can then use these modified JSONNET files to train a new model using AllenNLP.

If your model isn't one of the ones provided by PPB however, you'll have to change a not-insignificant portion of the project to use something like huggingface instead. Before you can really do that, you'll have to familiarize yourself with the way AllenNLP projects are organized. I will say this: AllenNLP makes writing projects and running experiments very easy, but it's a bit of a write-only paradigm as it can be challenging to understand how an AllenNLP project works when coming from the outside.

Luoyumeng commented 2 years ago

Thank you very much! Respect！

------------------ 原始邮件 ------------------ 发件人: "allenai/kb" @.>; 发送时间: 2021年10月4日(星期一) 下午5:04 @.>; @.**@.>; 主题: Re: [allenai/kb] How to inject knowledge into other pre-trained models (#37)

In theory, you should be able to specify a different model than BERT (as long as it's supported by pytorch-pretrained-bert) in the JSONNET files in the training_config subdirectory of this project. You can then use these modified JSONNET files to train a new model using AllenNLP.

If your model isn't one of the ones provided by PPB however, you'll have to change a not-insignificant portion of the project to use something like huggingface instead. Before you can really do that, you'll have to familiarize yourself with the way AllenNLP projects are organized. I will say this: AllenNLP makes writing projects and running experiments very easy, but it's a bit of a write-only paradigm as it can be challenging to understand how an AllenNLP project works when coming from the outside.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.