Question:Could I fine-tune the language model by classification task on another dataset?

allenai / bilm-tf

Tensorflow implementation of contextualized word representations from bi-directional language models

Apache License 2.0

1.62k stars 452 forks source link

Question:Could I fine-tune the language model by classification task on another dataset? #148

Closed gai1995 closed 5 years ago

gai1995 commented 5 years ago

I want to fine-tune the ELMo on a classification task and hope the parameters could change during the task. Could you give me some hints where to modify the code to suit the fine-tune training? I'm trying to modify the 'build_loss' in training.py, but stuck with how to feed my original sentence and class label...

Thanks!

matt-peters commented 5 years ago

Yes it is certainly possible to fine tune the model. To feed in your original sentence use the existing code in data.py to convert the words to sequences of character ids. You will need to write some additional code to convert your class label to ids.