Open shivance opened 2 weeks ago
@shivance thanks for your interest!
See: https://github.com/vllm-project/vllm/issues/7366 for a detailed discussion of enc-dec support. We would definitely welcome help on T5. For T5, the key is that we need an attention implementation with the custom bias
🚀 The feature, motivation and pitch
Hi vLLM team, As you already know, T5 is a perfect model and as mentioned in #7366, the project intends to add support for T5. I want to help the cause and contribute T5 to the project.
As this is my first contribution to vLLM project I am quite pumped up as well as eager to learn from you all.
Alternatives
No response
Additional context
No response
Before submitting a new issue...