-
**Describe the bug**
Hello. I tried to use a method to set the device_id, even if not documented. Sadly it works only on device 0
**To Reproduce**
This works
```python
import onnxruntime_genai …
-
### Describe the issue
I have a onnx model whose size is only 204.57MB，but when I create the session, gpu memory consumpation comes 1.16GB, when inferencing, the gpu memory consumpation comes to 2.25…
-
The code: it first loads Roberta base, prepares input and convert model to onnx and the load and run it,
import torch
import time
from transformers import RobertaTokenizerFast,Roberta…
Oxi84 updated
1 month ago
-
keep getting this error and I cannot bypass it
-
```dockerfile
FROM runpod/pytorch:2.2.1-py3.10-cuda12.1.1-devel-ubuntu22.04
```
using this docker file and running
```python
import inference.models.yolo_world.yolo_world
YOLO = inference.m…
-
I tested inswapper_128.onnx in GPU mode (nvidia V100, cuda: 11.8, cudnn:8.9.1, onnxruntime-gpu:1.15.1) and CPU mode respectively, and found it ran faster in CPU mode, did you test in GPU mode ?
-
**Describe the bug**
I won't get the GPU to get utilized on my MacBook.
Other apps like LLM can utilize up to 70 GB RAM for the graphic processor.
**To Reproduce**
Steps to reproduce the behavi…
-
The defalt cuda version for onnxruntim-gpu from pypi is cuda11.8. But I want to install the cuda12.x version. So I add extra url in my`project.toml` like
```
[[tool.rye.sources]]
name = "onnxruntim…
-
### First, confirm
- [X] I have read the [instruction](https://github.com/Gourieff/sd-webui-reactor/blob/main/README.md) carefully
- [X] I have searched the existing issues
- [X] I have updated the e…
-
Installed first
```
(PolyMind) PS D:\AI\PolyMind> pip install onnxruntime-gpu
Requirement already satisfied: onnxruntime-gpu in c:\users\alok\miniconda3\envs\polymind\lib\site-packages (1.16.3…