-
### System Info
A10
tensorrt-cu12-10.2.0.post1
tensorrt-cu12-bindings-10.2.0.post1
tensorrt-cu12-libs-10.2.0.post1
tensorrt_llm-0.12.0.dev2024072300
python==3.10
### Who can help?
@Tracin
…
-
I ran code below using WSL ubuntu in windows:
```
docker run -p 9090:9090 --runtime=nvidia --gpus all --entrypoint /bin/bash -it ghcr.io/collabora/whisperlive-tensorrt
# Build tiny.en engine
bas…
-
### System Info
just a simple python bug, system agnostic
### Who can help?
@byshiue
### Information
- [X] The official example scripts
- [ ] My own modified scripts
### Tasks
- [X] An officia…
-
### System Info
* GPU V100, A100
* docker image `nvidia/cuda:12.1.0-devel-ubuntu22.04`
* tensorrt-llm `0.9.0.dev2024020600`
### Who can help?
@byshiue
### Information
- [X] The offici…
-
Hello,
I have pretrained a model with huggingface and attempted to deploy it using the TRTLLM-Triton Server method as documented [here](https://github.com/k2-fsa/sherpa/blob/master/triton/whisper/mod…
-
### System Info
- CPU: x86_64
- CPU memory size: 96GB
- GPU: A100
- Tensorrt tag: latest one(https://github.com/NVIDIA/TensorRT-LLM/commit/5d8ca2faf74c494f220c8f71130340b513eea9a9)
- TensorRT v…
-
### System Info
I have pretrained whisper-large-v2 model with my custom dataset, and tried to build tensorrt-llm.
But I got `[Errno 2] No such file or directory: '/workspace/models/whisper-large-v…
-
**Describe the solution you'd like**
With the recent [TensorRT-LLM support for Whipser](https://github.com/NVIDIA/TensorRT-LLM/tree/main/examples/whisper), and now that PyTriton supports TensorRT-LLM…
-
Hi there,
I have been finetuning whisper models using huggingface. Further to convert the model to TensorRT_LLM format, i use a HF script that converts the models from its HF format to the original …
-
Been following the instruction at `https://github.com/k2-fsa/sherpa/tree/master/triton/whisper` and somehow running the line ```python3 build.py --output_dir whisper_large_v3 --use_gpt_attention_plugi…