boun-tabi-LMG / turkish-lm-tuner

Turkish LM Tuner
https://boun-tabi-lmg.github.io/turkish-lm-tuner/
MIT License
73 stars 6 forks source link

Fix Token Classification Integration #46

Closed gokceuludogan closed 5 months ago

gokceuludogan commented 5 months ago

This pull request addresses the integration of the token classification model of Turna encoder and BERTurk.

Changes:

The implementation was tested on BERTurk and mT5-small with the following commands:

python finetune.py --config-name ner_wikiann_token model_name=google/mt5-small
python finetune.py --config-name ner_wikiann_token model_name=dbmdz/bert-base-turkish-cased
gokceuludogan commented 5 months ago

POSDatasets need to be updated as well.