boun-tabi-LMG / turkish-lm-tuner

Turkish LM Tuner
https://boun-tabi-lmg.github.io/turkish-lm-tuner/
MIT License
73 stars 6 forks source link

Implement Turna Encoder For QA #48

Open gokceuludogan opened 5 months ago

gokceuludogan commented 5 months ago

To compare Turna with encoder-only models, Turna encoder and BERTurk, the implementation needs to be updated to support Question-Answering (QA):

Refer to the following link for the implementation process: https://github.com/huggingface/transformers/blob/f1185a4a73a03d238afce1b40456588d22520dd2/src/transformers/models/bert/modeling_bert.py#L1799.

You can check this guide for preprocessing, postprocessing, and custom training parameters required for QA: https://huggingface.co/docs/transformers/tasks/question_answering.