-
I'm using the `NNsight` class to wrap a 4-bit quantized LLaVA model and encountered this error:
```
...
File ~/miniconda3/envs/llava/lib/python3.10/site-packages/bitsandbytes/nn/modules.py:429, i…
-
Hi guys
I had a report earlier today from a user telling me that he tried one of my new AWQ models, and got an error indicating that only float16 is supported with AWQ.
I tested it myself with t…
-
我在docker中编译cpu版的paddle,遇到如下错误:
```
paddle/fluid/jit/CMakeFiles/jit_download_program.dir/build.make:57: recipe for target 'paddle/fluid/jit/CMakeFiles/jit_download_program' failed
make[2]: *** [padd…
-
I have searched in several places and was unable to pin down this issue.
i wanted to start a ticket before I do any more changes and experiments to my setup.
That way I can come back here and say wh…
-
Hi, I met the following error when I tried to load a llama model:
```
Loading checkpoint shards: 100%|████████████████████████████████████████████████████████████████████████████████████████████████…
ghost updated
5 months ago
-
Hi.
I run into this error when trying to fine-tune **Phi3 small**:
```triton.runtime.autotuner.OutOfResources: out of resource: shared memory, Required: 180224, Hardware limit: 101376. Reducing bloc…
-
### System Info
transformers==4.31.0
accelerate==0.21.0
deepspeed==0.13.2
bitsandbytes==0.42.0
### Who can help?
_No response_
### Information
- [X] The official example scripts
- [X] My own m…
-
### System Info
System Info
bitsandbytes 0.43.1
Python 3.10.12
"CUDA" library: rocm-libs Version: 6.0.0.60000-91~22.04
Ubuntu 22.04.1
Getting the following error after Mistral safetensors are …
-
I am trying to finetune a bnb quantized model for summarization using LORA
base_model - `cognitivecomputations/dolphin-2.2.1-mistral-7b`
I am training it for 1 epoch, the loss weirdly is at 0 from…
-
### System Info / 系統信息
ubuntu22
conda
python3.11
nvidia-cudnn-cu12
torch 2.3.0
vllm 0.5.0.post1
vllm-flash-attn 2.5.9…