documentation - flan-t5

axolotl-ai-cloud / axolotl

Go ahead and axolotl questions

https://axolotl-ai-cloud.github.io/axolotl/

Apache License 2.0

7.91k stars 869 forks source link

Open winglian opened 1 year ago

winglian commented 1 year ago

regular sft and lora sft are supported , but only on the q and k modules.

NanoCode012 commented 1 year ago

What does sft stand for?

winglian commented 1 year ago

supervised fine tuning

Nima-Nilchian commented 7 months ago

is or will be the aya model which comes from mt5, supported for finetuning? @winglian

Respaired commented 5 months ago

i'm very interested in this. but can't find any scripts in the 'examples'. enc-dec models have huge untapped potential.