-
-
I'm getting this error and I don't know what to do
-
OS: W10 LTSC
RAM: 16gb
GPU: Nvidia 3060 12gb VRAM
I keep running into OOM errors when i try to use a single LoRA:
Windows 11 X64
64GB RAM
GPU：2070S 8GB VRAM
I dont know Why.
[flux1-dev-Q4_0…
-
I have encountered a problem when using QLoRa from the LoRALinear class you have in Torchtune. Apparently, when the ‘quantize_base’ parameter is set to True, the NF4Tensor class is called, which trans…
-
Hi @minuszoneAI
Can you add some node support loading lora when using Marlin model
Thank you in advance
-
Add bnb Linear8 and Linear4
-
### System Info
I am using a Tesla T4 16 gb
### Reproduction
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
base_model_id = "mistralai/Mistral-7B-…
-
Did your FLUX easy loader support the new loader Bitsandbyte it loads Flux super-fast and I'm able to use it with 6gb V-Ram
Here is the link to models and implementation:
https://github.com/comfyano…
-
Since ba01ad37, LoRas loaded in 8bit to the Q8_0 GGUF generate to a poor quality. Loading the LoRa in 16bit appears to fix this issue, but there are subtle differences in the generations from rounding…