-
Hi all, this issue will track the feature requests you've made to TensorRT-LLM & provide a place to see what TRT-LLM is currently working on.
Last update: `Jan 14th, 2024`
🚀 = in development
#…
-
I found that in the benchmark/suite has the output time to first token. However, when I run `python benchmark.py --model meta-llama/Llama-2-7b-hf static --isl 128 --osl 128 --batch 1` an error occurs:…
-
Thanks for your really impressive work.
I was wondering how to extract the token probability with TensorRT (a little bit lit what you did in this example with ctranslate2)
```
import whisper_s2…
-
Hi,
while trying to run this
```
python build.py --model_dir $model_dir$ \
--dtype float16 \
--use_gpt_attentionZ_plugin float16 \
--use_gemm_…
-
Any instructions for installing this without needing Docker?
I setup a new Python environment with these commands
```
python -m pip install --upgrade pip
pip install --no-cache-dir --ignore-inst…
-
Thanks for putting so much work into this, its so polished already!
Just want to understand if speaker diarization is something planned in the future?
Thanks!
-
Greetings.
I followed the exact instructions for the 3090 image.
However, I am getting these problems when I open the browser and try to start audio capturing:
```
nginx-1 | 172.1…
-
Hello,
Will this run on OSX Metal or CoreML or mlx frameworks or does it require the CUDA framework?
Thank you.
-
Just did a standard `PyPi` download in my venv as per
`pip install pywhispercpp`
A standard script with:
```python
import pywhispercpp.model as m
modelPath: str = ...
filePath: str = ..…
-
I try to run:
```
import whisper_s2t
model = whisper_s2t.load_model(model_identifier="large-v3", backend='TensorRT-LLM', asr_options={'word_timestamps': True})
files = ['1.wav']
lang_codes = …