-
*TL;DR: Please add support for AMD GPUs on Linux through ROCm.*
Greetings,
I was wondering if it is being considered to support ROCm for inference? Since an RX 7900 XTX is currently the only option …
-
**UPD:** the missing `std::shared_ptr` support context is in the message https://github.com/mono/CppSharp/issues/1860#issuecomment-2297731111. Currently this support seems missing, and `shared_ptr`/`u…
-
Can someone help me. Why am I getting this error when I run inference:
Traceback (most recent call last):
File "/data/lwq/openfold-1.0.0/lib/conda/envs/openfold_venv/lib/python3.7/runpy.py", lin…
-
**Description**
I am deploying a YOLOv8 model for object-detection using Triton with ONNX backend on Kubernetes. I have experienced significant CPU throttling in the sidecar container ("queue-proxy")…
-
**Description**
Could not load model using mlflow with minIO as model repository. I have tried this AWS S3 bucket and it worked as expected. have followed this article [MLflow Triton Plugin](https://…
-
using pyenv + venv + Docker, llama stack run failed and seems cannot found model directory
```
$ llama stack run my-local-stack
+ '[' -n '' ']'
+ '[' -z '' ']'
+ docker run -it -p 5000:5000 -v …
-
### System Info
cargo version
cargo 1.80.1 (376290515 2024-07-16)
Haven't been able to run the docker file to get more details..
I am trying to run the docker on CPU
### Information
- [X] Docke…
-
### Search before asking
- [X] I have searched the YOLOv8 [issues](https://github.com/ultralytics/ultralytics/issues) and [discussions](https://github.com/ultralytics/ultralytics/discussions) and fou…
-
### The bug
English is not a native language. There are about 55,000 objects in the shared album. In the mobile app, opening an album takes about a minute. Once opened it works quickly. If you leave …
-
Hi,
Is there any tutorial that we can refer to so that we could serve a deberta model using fastertransformer in Triton?
I think the steps would be:
1. Convert a deberta-v2 model into fastertrans…