search

bigscience-workshop / xmtf

Crosslingual Generalization through Multitask Finetuning
https://arxiv.org/abs/2211.01786
Apache License 2.0
513 stars 37 forks source link

How to fineutne mT0 with specific down-stream data? #12

Open benchen4395 opened 1 year ago

benchen4395 commented 1 year ago

Can you provide the continue fine-tune code for mT0 to specific down-stream data task，we want to test it for specific scene, e.g. retrieval and recommendation. We find a similar version for continue fine-tuning flan-t5 in https://www.philschmid.de/fine-tune-flan-t5-deepspeed. Is it same to xmtf, or can you provide an official example, just like classification or QA? Best wishes.

Muennighoff commented 1 year ago

We just fine-tune on all of xP3 - you can find the guide here: https://github.com/google-research/t5x/blob/main/docs/usage/finetune.md

Let me know if you run into any problems!

MrSchnappi commented 1 year ago

Hi. I download the mt0-xxl-mt, but I don't know how to use，Can you provide the guide ？

Muennighoff commented 1 year ago

There's a section with code snippets depending on your hardware here: https://huggingface.co/bigscience/mt0-xxl-mt#how-to-use