-
### Describe the issue
can you help me with this, i don t have a clue
### To reproduce…
-
### 🐛 Describe the bug
Right now Inductor API offers no way queuing whether given DSO target CUDA or CPU
And attempts to load CUDA binaries in CPU corrupts CUDA context, so trick like
```cpp
t…
-
Hello manyoso,
I obtained the following error messages during installation of https://developer.nvidia.com/cuda-downloads?target_os=Windows&target_arch=x86_64 ( NVIDIA CUDA 10.1 Toolkit…
-
### Describe the issue
use shape_inference.quant_pre_process to preprocess will result in error even if i set skip_optimization=True
, I met the following error:
```
linux-x86_64-gpu/opencv-4.3.0/modules/core/src/cuda/gpu_mat.cu:121: error: (-217:Gpu API call) CUDA d…
-
### Checklist
- [ ] The issue exists after disabling all extensions
- [X] The issue exists on a clean installation of webui
- [X] The issue is caused by an extension, but I believe it is caused b…
-
Hi!
I want to try cricket for C/R in cpu mode (no in-kernel checkpointing). However, when I run restore it fails with segfault.
```
(gdb) r
Starting program: /home/alexndrfrolov/cricket/cpu/c…
-
Since vLLM 0.2.5, we can't even run llama-2 70B 4bit AWQ on 4*A10G anymore, have to use old vLLM. Similar problems even trying to be two 7b models on 80B A100.
For small models, like 7b with 4k to…
-
### Describe the issue as clearly as possible:
When using outlines with the Llama 3.2 Vision model, simple regex pattern generation works, but JSON schema-based generation fails with index out of bou…
-
### System Info
Version: v.1.4.0
Cargo version: cargo 1.79.0 (ffa9cf99a 2024-06-03)
GCC version: 11.4.1
GPU: Compile with CUDA_COMPUTE_CAP=86 on machine without GPU (but with CUDA 12.1).
I plan t…