Open nshen7 opened 1 month ago
Have you updated transformers
to the newest version? @nshen7
I was using Version 4.41.2, but the error persists with the newest version.
To add to the context, I was using the default tokenizer from RLHFlow/ArmoRM-Llama3-8B-v0.1
tokenizer = AutoTokenizer.from_pretrained(CFG.MODEL_NAME, trust_remote_code=CFG.TRUEST_REMOTE)
INPUT_IDS_A = tokenizer.apply_chat_template(
train['message_a'].tolist(),
tokenize=True,
padding=True,
truncation=True,
max_length=CFG.MAX_LENGTH,
return_tensors='np'
)
INPUT_IDS_B = tokenizer.apply_chat_template(
train['message_b'].tolist(),
tokenize=True,
padding=True,
truncation=True,
max_length=CFG.MAX_LENGTH,
return_tensors='np'
)
And I was trying to fine-tune the gating layer of the model with LoRA:
lora_config = LoraConfig(
r=CFG.LORA_RANK, # the dimension of the low-rank matrices
lora_alpha = CFG.LORA_ALPHA, # scaling factor for LoRA activations vs pre-trained weight activations
lora_dropout= CFG.DROPOUT,
bias='none',
inference_mode=False,
task_type=TaskType.SEQ_CLS,
target_modules=CFG.LORA_MODULES ) # Only Use Output and Values Projection
model = get_peft_model(base_model, lora_config)
Configs:
class CFG:
NUM_EPOCHS = 1
BATCH_SIZE = 16
DROPOUT = 0.05
MODEL_NAME = "RLHFlow/ArmoRM-Llama3-8B-v0.1"
TRUEST_REMOTE = True
SEED = 2024
MAX_LENGTH = 1024
NUM_WARMUP_STEPS = 128
LR_MAX = 5e-5
NUM_LABELS = 3
GATING_TEMP = 10
LORA_RANK = 4
LORA_ALPHA = 8
LORA_MODULES = ["gating.layers.0", "gating.layers.1", "gating.layers.2", "gating.layers.3"]
Same error.
I believe the reason is about the function defined in the modeling_custom.py, find_token_for_gating()
, which caputures the token partten:
<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n
The deeper reason lies on the dataset side. I find the code always falls on a sample with the "response" are null. For example the sample with prompt_id c0dfe114bb80a25990c193539a0e8e43557ba7236fd00e71731b852b4e7849a9
in the UltraFeedback_binarized dataset:
https://huggingface.co/datasets/HuggingFaceH4/ultrafeedback_binarized/viewer/default/train_prefs?q=Maybe+using+FILTER+or+VLOOKUP&row=24707
I believe the solution lies on removing these bad samples.
Hi there,
I got this "Token pattern not found in the list" error when I tried out the model under no_grad() condition. Would you take a look at this please? Many thanks!! See below for the code and error message: