AI4Bharat / IndicTrans2

Translation models for 22 scheduled languages of India
https://ai4bharat.iitm.ac.in/indic-trans2
MIT License
214 stars 59 forks source link

Fine-tuning with LoRA doesn't work with training data structured as described! #93

Closed micko93ns closed 1 month ago

micko93ns commented 1 month ago

Hi everyone. Nice project 🚀

I am trying to fine tune models following readme. After I structured the data as described, load_and_process_translation_dataset doesn't work as expected. It errors with FileNotFoundError: Source (train.eng_Latn) or Target (train.ory_Orya) file not found in en-indic-exp/.

Additionally command line argument for model name in train_lora.sh is model_name while train_lora.py expect model.

Thank you!