Smaller T5_model

[seanphan]

Incredible work, thank you so much.

Instead of the original T5 model I find this also working https://huggingface.co/ybelkada/flan-t5-xl-sharded-bf16/tree/main

6GB vs 10GB.

Maybe you can clone a repo of T5 that has minimum requirements and can auto install with pip?

[kijai]

It only needs the encoder, so it can be halved from that: https://huggingface.co/Kijai/flan-t5-xl-encoder-only-bf16/tree/main

[wangfeng35]

It only needs the encoder, so it can be halved from that: https://huggingface.co/Kijai/flan-t5-xl-encoder-only-bf16/tree/main

You rock!!! It saves a lot of space!

[kijai]

They seem to have official one too here: https://huggingface.co/QQGYLab/ELLA/tree/main/models--google--flan-t5-xl--text_encoder

[seanphan]

@kijai awesome !!

[JettHu]

They seem to have official one too here: https://huggingface.co/QQGYLab/ELLA/tree/main/models--google--flan-t5-xl--text_encoder

Yeah, the encoder only t5-xl was just pushed on hf hub. Try this one. @seanphan

Of course, you can also use the bf16 version if your device support bf16.

[seanphan]

Thank you !! @JettHu

[JettHu]

close it.