search

bentoml / openllm-models

13 stars 8 forks source link

[Feature] How about to add flan-t5 series models #30

Open YuriyGavrilov opened 2 weeks ago

YuriyGavrilov commented 2 weeks ago

Just about to add this models to list.

aarnphm commented 2 weeks ago

vllm support encoder-decoder models, but not flan-t5 afaict.

tbh SLM are a lot better now and outperform t5, such as phi3, and gemma would suffice

bojiang commented 1 week ago

It seems T5 are supported by llamacpp. https://github.com/ggerganov/llama.cpp/pull/8141 @aarnphm Any idea?

aarnphm commented 1 week ago

I mean we can add support for t5,

my feeling is for encoder-decoder models there must be better option?