Open ByUnal opened 7 months ago
I've found that the problem here is Quantization:
import torch
from transformers import BitsAndBytesConfig
config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_quant_type="nf4",
# llm_int8_has_fp16_weight = True,
bnb_4bit_use_double_quant=True,
bnb_4bit_compute_dtype=torch.bfloat16,
)
model = AutoModelForCausalLM.from_pretrained(model_name,
trust_remote_code=True,
quantization_config=config,
device_map="auto"
)
Model gives the error above, but it works without quantization.
I'm trying to train
RWKV/rwkv-4-world-430m
with LoRA by using Transformer's Trainer module. I've performed chunking to my data (chunk size = 128), and I initiated the training. Training continues properly, but it throws an error at the end of the epoch just before evaluating. The error is:KeyError: "attribute 'weight' already exists"
Here is my training code and full error:
Code:
Full of error:
Please help me on this. Could be related with LoRa ?