-
### Search before asking
- [X] I have searched the Inference [issues](https://github.com/roboflow/inference/issues) and found no similar bug report.
### Bug
`pip install inference` results in:
…
-
[LMDeploy](https://github.com/InternLM/lmdeploy), as an AI deployment platform supporting multiple backend services, has always been committed to providing fast and stable AI model deployment services…
-
### 🚀 The feature, motivation and pitch
AOT inductor looks like the upcoming means to do inference from native code that was trained in pytorch, and the replacement for torchcript export to native co…
-
### Search before asking
- [X] I have searched the YOLOv8 [issues](https://github.com/ultralytics/ultralytics/issues) and [discussions](https://github.com/ultralytics/ultralytics/discussions) and fou…
-
### Describe the issue
onnxruntime + openvino need double memory compared with openvino-only
I guess the onnx model and opnevino model are in the memory in the same time.
my model size: 330M
onnx…
-
### Reminder
- [X] I have read the README and searched the existing issues.
### Reproduction
Hi There, I am observing a difference in output between llama factory inference and llama.cpp.
I am…
anidh updated
1 month ago
-
### Describe the issue
Following the documentation, I dynamically quantiized a resnet based model. The model is quantized and saved without error. However, when I try to create an inference session u…
-
Hi,
Just to check if I set up my machine with a MI100 GPU correctly I ran the "AI Benchmark" from https://ai-benchmark.com/ranking_deeplearning_detailed.html .
The inference speed is pretty good, …
Epliz updated
1 month ago
-
Trying to run any of the Pygame Zero examples fails with
```
PROBLEM IN THONNY'S BACK-END: Exception while handling 'Run' (ValueError: AST node line range (1000000, 1) is not valid).
See Thonny's b…
-
### Describe the issue
Hello,
I am working with a Jetson Orin Nano from NVIDIA and I am trying to execute an inference with onnxruntime with a onnx model that was converted from pytorch to onnx.
…