-
### Your current environment
vllm 0.5.3.post1
vllm-flash-attn 2.5.9.post1
### 🐛 Describe the bug
(agiclass) root@autodl-container-c9174bac52-9e5578…
-
I am running CrisisFACT.ipynb notebook on Colab on Python v3.9.16
Used colab's fallback runtime version.
I am getting error while importing monoT5 from pygaggle. Kindly help me resolve the issue …
-
Would it be possible to create a [Triton backend](https://github.com/triton-inference-server/backend) from this implementation?
> A Triton backend is the implementation that executes a model. A bac…
-
We need to make sure that:
- [x] CUDA
- [x] pypi binaries with slimmed dependencies are usable in standard AWS containers (amazonlinux:2 regression in 1.13) - @PaliC
```pip3 install torch==2…
-
### Problem Description
Seeing ~15 PyTorch UTs failures at TOT triton-mlir reporting this failure previously hidden by https://github.com/ROCm/triton/issues/412
```
FAILED [0.1121s] test_torchi…
-
When I test the model I trained on the app, I encountered the following error. How can I solve this? However in the training stage, there is no error.
```python
Traceback (most recent call last):
…
-
I'm encountering an error when running kernels on some machines.
It is very sensitive to the exact kernel code that's written. Even trivial changes such as trimming whitespace or adding/removing c…
-
When attempting to repair the wheel for `triton_nightly` to the `manylinux2014_x86_64` ABI, the auditwheel tool fails with the following error:
```
auditwheel: error: cannot repair "/tmp/cibuildwh…
-
I encountered the following problem when finetuning the model with the guidance of README.md.
## Here is the detailed error:
(alpaca) root@iZwz95ccn6prjs8ioz8bbdZ:/data/stanford_alpaca# sh order.s…
-
### Your current environment
H100 (but I believe it happens in any machine)
### 🐛 Describe the bug
```
--enable-chunked-prefill --num-max-batched-tokens 2048 --kv-cache-dtype "fp8"
```
S…