-
Is there any way we can save the model with the registered custom ops, so that each time when we load the onnx model we don't have to register the custom ops? Right now every time we load the model, w…
-
**Is your feature request related to a problem? Please describe.**
For now the Tensorflow and ONNX backends in Triton support thread controls ([here](https://github.com/triton-inference-server/tens…
-
**Description**
Caching is not woring with ensemble models.
**Triton Information**
23.07
**Are you using the Triton container or did you build it yourself?**
Triton container
**To Reproduce*…
-
i have 2 x a100 gpus,
i hava been training one task on gpu1,
and i want to train another tasks on gpu2 at the same time,
but i get error as followings:
```
CUDA_VISIBLE_DEVICES=1 \
xtuner t…
-
First, thanks for creating this great and high performant framework! I've looked in the open and closed issues and couldn't find this one.
## Description
It would be really cool to be able to enabl…
-
hi everyone
i runing tritonserver vllm and i want runing with dynamic batching, but i encountered an error. It seems like it has something to do with my input
Inference with curl:
curl -X POST loca…
-
I have a conformer CTC model built with the NeMo framework (https://github.com/NVIDIA/NeMo), which can be normally converted and deployed with Riva 2.11.0. However, if I convert the same NeMo file to …
-
Hey! You have a wonderful project. Tell me, if possible, how to run the example "Calculating the speed of cars using YOLO v4 in real time" and other examples in this repository in multi-camera mode. I…
-
Using version `accel-ppp version 1.12.0-149-gff91c73`
The function `reload_exec` can cause `stack-buffer-underflow`:
Here is the asan report:
```
============================================…
-
[RFD27/Container Monitor](https://github.com/joyent/rfd/blob/master/rfd/0027/README.md) integration requires two things:
1. TLS certs based on a user's SSH key
2. Discovery of RFD27 endpoints
### Auth…