-
Hello,
so our current models stack consists of a set of models built in TensorRT and the whisper ASR model.
I'd like to use triton server to host all these models. Since whisper can be con…
-
I noticed that the latest benchmark from whisper.cpp is from February 2, 2023 and presumably the others are equally outdated. Can someone please do updated benchmarks for the three backends tested? …
-
code
`model = whisper_s2t.load_model(model_identifier="large-v2", asr_options={'word_timestamps': True},backend='TensorRT-LLM')
files = ['output.wav']
lang_codes = ['en']
tasks = ['transcribe']…
-
Why does the whisper model need 17GB of video memory? fast-whipser only needs 4G video memory? And I haven't found a way for whisper to quantize Int. Is it not supported now? This video memory occupie…
-
### System Info
- CPU architecture: x86_64
- CPU/Host memory size: 32GB DDR4
- GPU properties
- GPU name: RTX 3070 Ti
- GPU memory size: 8GB
- Libraries
- TensorRT-LLM version: 0.12.0.d…
-
### System Info
GPU: H20 server
CUDA Version: 12.5
Driver: 555.42.02
TRTLLM Commit: 2d234357c6e69fa514f6e9b4d4a5ad3bc431c4a6
built from source on linux
### Who can help?
_No response_
…
-
### System Info
x86_64
755G
nvidia T4
ubuntu 22.04
trtllm version : https://github.com/NVIDIA/TensorRT-LLM/archive/9691e12bce7ae1c126c435a049eb516eb119486c.zip
pip install tensorrt-llm==0.11…
-
Hello. This project looks very interesting. I hit some issues building the Dockerfile as described in the readme:
```
docker build -t wordcab-transcribe:latest .
docker run -d --name wordcab-tr…
-
-
Hi all, this issue will track the feature requests you've made to TensorRT-LLM & provide a place to see what TRT-LLM is currently working on.
Last update: `Jan 14th, 2024`
🚀 = in development
#…