-
**Please describe the feature you want**
I've been using a large completion model with my GPU. I'd like to add a chat model as well, but there's not enough GPU memory for the large completion model…
-
I attempted to run benchmarks for the llama-3-8b-instruct and llama-3.1-8b-instruct models using both CPU and GPU, but the process failed. (I successfully tested the llama2-7b-chatbot model)
I f…
-
Would it be possible to run the models on the CPU, in order to avoid the error:
> RuntimeError: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver …
-
I'm noticing with v0.3.2 my CPU is getting slaughtered. The UI revamp is worse than the previous iteration with GPU offload now hidden on "My Models" page but even with all the layers assigned to GPU …
-
### 🐛 Describe the bug
###
Following code runs successful in PyTorch 2.1.1
However in PyTorch 2.4, the hook will not be called
```
import os
import torch
import torch.nn as nn
import torc…
-
torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 56.50 GiB.
V100 32G
5B model, `enable_model_cpu_offload()` option and `pipe.vae.enable_tiling()` optimization were enabled
using …
-
### Search before asking
- [X] I have searched the Ultralytics YOLO [issues](https://github.com/ultralytics/ultralytics/issues) and found no similar bug report.
### Ultralytics YOLO Component
Pred…
-
### Description of the bug | 错误描述
这么设置的:
{
"bucket_info":{
"bucket-name-1":["ak", "sk", "endpoint"],
"bucket-name-2":["ak", "sk", "endpoint"]
},
"models-dir":"D:/too…
-
### 🐛 Describe the bug
JIT tracing a quantized model that has forward_pre_hooks throws the following error:
`RuntimeError: Couldn't find method: 'forward' on class: '__torch__.torch.ao.nn.intri…
-
```julia
ERROR: MethodError: ExaCore(::Type{Float64}, ::CPU) is ambiguous.
Candidates:
ExaCore(::Type{T}, backend) where T