-
Is there a way to further convert the model to tensorRT? I think the model speed could be improved.
-
### Issues Policy acknowledgement
- [X] I have read and agree to submit bug reports in accordance with the [issues policy](https://www.github.com/mlflow/mlflow/blob/master/ISSUE_POLICY.md)
### Willi…
-
some onnxruntime errors happend
```
[ 72%] Building CXX object sherpa-onnx/csrc/CMakeFiles/sherpa-onnx-microphone-offline.dir/microphone.cc.o
../../_deps/onnxruntime-src/lib/libonnxruntime_provider…
-
Here is a cleaned up GitHub issue request:
# Long context inference overhead with Llama NeuronX
I followed the [Llama NeuronX tutorial](https://pytorch.org/blog/high-performance-llama/) to host …
-
Here is a cleaned up GitHub issue request:
# Long context inference overhead with Llama NeuronX
I followed the [Llama NeuronX tutorial](https://pytorch.org/blog/high-performance-llama/) to host …
-
I am trying to run `on_device_training/mobile/ios` on an M1 Mac. I could not install `onnxruntime-training`.
Inspecting the verbose output of Pip, it seems there exists no `.whl` for ARM. I've trie…
-
I wanna use triton+tensorrt to deploy whisper, a transformer-like arc asr model.
I wanna use kv-cache to accelerate inference speed, so i use python-backend and dlpack to do this
when i build a d…
-
First of all, thank you for maintaining this tool.
Will it work in Turkish ?
Regards.
-
**Describe the bug**
A clear and concise description of what the bug is.
```python-traceback
Traceback (most recent call last)/tmp/ipykernel_98/3684188953.py in
10 audio_file=audio_fil…
-
## Description
Met this error when converting my onnx model to tensorrt using new tensorrt 8.5.
Attached my onnx file.
## Environment
**TensorRT Version**: 8.5.1.7
**NVIDIA GPU**:
**N…