-
Hi there,
I have been finetuning whisper models using huggingface. Further to convert the model to TensorRT_LLM format, i use a HF script that converts the models from its HF format to the original …
-
Dynamic LoRA (Low-Rank Adaptation) switching functionality, allowing users to change LoRA models on-the-fly during inference without reloading the entire model.
-
**Describe the solution you'd like**
With the recent [TensorRT-LLM support for Whipser](https://github.com/NVIDIA/TensorRT-LLM/tree/main/examples/whisper), and now that PyTriton supports TensorRT-LLM…
-
### System Info
I have pretrained whisper-large-v2 model with my custom dataset, and tried to build tensorrt-llm.
But I got `[Errno 2] No such file or directory: '/workspace/models/whisper-large-v…
-
Been following the instruction at `https://github.com/k2-fsa/sherpa/tree/master/triton/whisper` and somehow running the line ```python3 build.py --output_dir whisper_large_v3 --use_gpt_attention_plugi…
-
### System Info
NVIDIA RTX A6000
### Who can help?
@juney-nvidia
Hi
I'm interested in using TensorRT-LLM for multiple inference inferences, but I'd like to be able to adjust the `num_be…
-
## Bug Description
https://github.com/pytorch/TensorRT/blob/main/examples/dynamo/mutable_torchtrt_module_example.py
I replaced hugging face whisper model instead of diffusion model
## To Repr…
-
hello!
I build int8 weights:
INFERENCE_PRECISION=float16
WEIGHT_ONLY_PRECISION=int8
MAX_BEAM_WIDTH=4
MAX_BATCH_SIZE=8
checkpoint_dir=whisper_large_v3_weights_${WEIGHT_ONLY_PRECISION}
output_dir…
-
Hey @shashikg great repo and cheers to the insane efforts in building this repo.
I have a finetuned whisper model (both in original openai and HF formats ) which I want to use in TensorRT backend …
-
Hi WhisperX Team,
I was wondering if you consider support https://github.com/NVIDIA/TensorRT-LLM/tree/main/examples/whisper the tensorrt-llm backend of whisper. I have done several benchmark test us…