search

dwadden / dygiepp

Span-based system for named entity, relation, and event extraction.
MIT License
569 stars 120 forks source link

Custom BertModel from scratch #103

Closed jbdel closed 2 years ago

jbdel commented 2 years ago

Hello,

Thank you for this amazing repo!

I understand bert_model is the name of a pretrained BERT model available on HuggingFace.

For an ablation study, I would like to train a simple BERT model, from scratch, with a custom config (like, say, 2 layers). Is there a way to make that work? Thank you very much for your reply.

dwadden commented 2 years ago

Hi,

Thanks for your question! The mechanism for interfacing between Huggingface transformers and AllenNLP is pretty complicated. I think your best bet is probably just post an un-trained two-layer transformer to Huggingface and point to that model in the config. The alternative of digging around in the guts of AllenNLP doesn't seem very attractive.

jbdel commented 2 years ago

its actually a smart work-around, thanks!

JB