is this script compatible with flan-t5?

[2533245542]

Is the training script compatible with flan-t5, like changing the model name to google/flan-t5-xxl, will it work?

On my task I found flan-t5 models work much better than llama and want to see if fine-tuning can improve it more.

Thanks

[kbressem]

This depends on what you want to do. Training in fp16/bf16 will probably work. For LoRA I don't know. You may need to specify other target layers in the model to make it work.