-
**Description**
Sometimes you have crashes like the ones below.
`ui_model_menu.py` is quite nice and dandy by reporting the python stack when it can, but as you can see below if you were only to …
-
- [onnxruntime](https://github.com/lyuwenyu/RT-DETR/blob/main/rtdetr_pytorch/tools/export_onnx.py#L84-L122)
- [TensorRT](https://github.com/lyuwenyu/RT-DETR/blob/main/benchmark/trtinfer.py)
- [RT-DE…
-
Inference failed while using ab with more than 3 concurrency but was ok with 1 or 2 concurrency. Using an A10G GPU, with Driver Version: 545.23.06，CUDA Version: 12.3， trt version：9.1， vicuna 13b-1.5-…
-
### 🐛 Describe the bug
```
import os, sys
import torch
from functools import lru_cache, partial
from torch.nn.attention.flex_attention import (
_DEFAULT_SPARSE_BLOCK_SIZE,
create_bl…
-
### Checklist
- [ ] 1. I have searched related issues but cannot get the expected help.
- [ ] 2. The bug has not been fixed in the latest version.
### Describe the bug
Hello, after successful quant…
-
Greetings,
I'm getting this error when use ml backend with PyTorch.
`RuntimeError: Cannot re-initialize CUDA in forked subprocess. To use CUDA with multiprocessing, you must use the 'spawn' star…
-
### 🐛 Describe the bug
Hello, I'm following [this tutorial](https://pytorch.org/executorch/stable/build-run-qualcomm-ai-engine-direct-backend.html) to try and get it running on my phone. The only cha…
-
### 🐛 Describe the bug
Running the following:
```python
import torchaudio
from pathlib import Path
test_audio_path = Path('test.wav')
torchaudio.load(test_audio_path)
```
Produces the fo…
-
**Description**
A clear and concise description of what the bug is.
I am trying to use the newly introduced [triton inference server In-Process python API](https://github.com/triton-inference-server…
-
**Describe the bug**
I run the following code
```cpp
#include "cudnn-frontend/include/cudnn_frontend.h"
namespace fe = cudnn_frontend;
int main()
{
cudnnHandle_t handle;
assert(cudnnCreate(…