-
### 🚀 The feature, motivation and pitch
Hi vLLM team,
As you already know, T5 is a perfect model and as mentioned in #7366, the project intends to add support for T5. I want to help the cause and co…
-
Unique ops in T5
- Eltwise
- multiply
- add
- sqrt
- subtract
- recip
- exp
- Reduce Avg
- Reduce Max
- Reduce Sum
- Reshape
- Matmul
- hslice
- hstack
- transpose
-
# Prerequisites
Please answer the following questions for yourself before submitting an issue.
- [ ] I am running the latest code. Development is very rapid so there are no tagged versions as of…
-
https://earth.bsc.es/gitlab/digital-twins/de_340-2/project_management/-/issues/337#note_305205
-
I tried to load a T5 model but it seems not supported.
```
---------------------------------------------------------------------------
NotImplementedError Traceback (most re…
-
Hello everyone,
First off, a big thanks to city96 for the awesome work they've been contributing to the community. It's been incredibly helpful!
Here are my system specs:
Processor: Intel i5-13…
-
**The bug**
Could you add support or provide some guidance (pun intended) so I can add support to the family of T5 model ?
**To Reproduce**
```python
import guidance
model_id = 'google/flan-t5-…
-
### System Info
Linux gaudi2-wsf-test 5.15.0-92-generic #102-Ubuntu SMP Wed Jan 10 09:33:48 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
docker image: ghcr.io/huggingface/tgi-gaudi:2.0.0
###…
-
This is taking about 2 hours with the smallest model.
I presume the issue is that my GPU cannot load a t5_XXL model into memory. According to the Huggingface page the model weights are 44.5 Gb.
…
-
### System Info
L4 GPU (AWS G6.12xl) with TensorRTLLM 0.11.0, running with Tritonbackends
### Who can help?
_No response_
### Information
- [ ] The official example scripts
- [ ] My own modified …