search

boun-tabi-LMG / turkish-lm-tuner

Turkish LM Tuner
https://boun-tabi-lmg.github.io/turkish-lm-tuner/
MIT License
73 stars 6 forks source link

PEFT integration/example #43

Open gokceuludogan opened 5 months ago

gokceuludogan commented 5 months ago

Given the model size being 1.1B parameters, I believe PEFT integration into the library would be beneficial, or at least having an example with our datasets and evaluators.

meliksahturker commented 5 months ago

I have prepared a sample notebook that contains fine-tuning + inference for the paraphrasing task. However the fine-tuned model re-produces the input most of the time so I am debugging that right now. Will push when we have an end-to-end notebook with fine-tuning, saving, loading and inference. @gokceuludogan fyi.