-
hi, i met a problem when onnx inference on gpu
1. onnx inference on gpu slower than onnx cpu inference much time and sometimes faster than gpu pt inference(2 times acceleration)
2. when i inference …
-
As we start onboarding more dtypes we ideally want them to work in as many different situations as possible so opening this tracker and will update the table as things change. If I should be adding mo…
-
### #
- [X] I have searched the existing issues
### Current behavior
I was playing with Jan for the first time and realised that GPU acceleration wasn't enabled.
I toggled the "GPU Acceleration" s…
-
$ python3 main.py
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
I0000 00:00:1729510536.107929 11848630 gl_context.cc:357] GL version: 2.1 (2.1 ATI-4.14.1), r…
becle updated
8 hours ago
-
- [ ] [Inference with Reference: Lossless Acceleration of Large Language Models by Nan Yang et al.](https://arxiv.org/abs/2304.04487)
# Inference with Reference: Lossless Acceleration of Large Langua…
-
`Traceback (most recent call last):
File "/home/code/hallo2/scripts/inference_long.py", line 35, in
import torch
ModuleNotFoundError: No module named 'torch'
(base) root@ecm-e6d2:/home/code…
-
Great job! I do inference on 3090, it takes about 0.7 seconds to calculate the time of two images, and what other operations can be used for inference acceleration, in addition to reducing the image r…
-
### Search before asking
- [X] I have searched the Inference [issues](https://github.com/roboflow/inference/issues) and found no similar bug report.
### Bug
If you install `inference-gpu` on a mac…
-
### The bug
The process crashes immediately after startup. Memory utilization reaches 8GB in a few seconds from the process and it stops working even on the CPU N5095
2024-10-22 11:55:32.645913931…
-
There is a number of issues with current TRT acceleration path in MONAI:
- For some networks it's only practical/possible to trace/export certain sub-module, like image_encoder. Current solution r…