search

HestiaSky / E4SRec

MIT License
35 stars 6 forks source link

Runtime error "Cannot set a non-string value as the PAD token" #4

Open nancheng58 opened 6 months ago

nancheng58 commented 6 months ago

Dear authors, Thanks for your nice work! I have already cloned this repo and downloaded the 'Platypus2-70B-instruct' model. However, I encounter (meet) with the following error when running the 'fine-turning.sh':

Traceback (most recent call last):
  File "finetune.py", line 245, in <module>
    fire.Fire(train)
  File "/data/anaconda3/envs/llama/lib/python3.8/site-packages/fire/core.py", line 141, in Fire
    component_trace = _Fire(component, args, parsed_flag_args, context, name)
  File "/data/anaconda3/envs/llama/lib/python3.8/site-packages/fire/core.py", line 475, in _Fire
    component, remaining_args = _CallAndUpdateTrace(
  File "/data/anaconda3/envs/llama/lib/python3.8/site-packages/fire/core.py", line 691, in _CallAndUpdateTrace
    component = fn(*varargs, **kwargs)
  File "finetune.py", line 118, in train
    model = LLM4Rec(
  File "/data/baseline/E4SRec/model.py", line 42, in __init__
    self.llama_tokenizer.pad_token = 0
  File "/data/anaconda3/envs/llama/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 1145, in pad_token
    raise ValueError("Cannot set a non-string value as the PAD token")
ValueError: Cannot set a non-string value as the PAD token

It seems like the token cannot be the number.

nancheng58 commented 6 months ago

I replaced the PAD token from 0 to '0' and solved this issue. Due to the resourse restricted, I replaced the basemodel to Llama2-13B to train and evaluate the E4SRec, but the gained result is very poor. The finetune script is

torchrun --nproc_per_node=8 --master_port=1907 finetune.py \
    --base_model /data06/Llama-2-13b-hf \
    --data_path Beauty \
    --task_type sequential \
    --output_dir ./LLM4Rec \
    --batch_size 16 \
    --micro_batch_size 1 \
    --num_epochs 1 \
    --learning_rate 0.0003 \
    --cutoff_len 4096 \
    --val_set_size 0 \
    --lora_r 16 \
    --lora_alpha 16 \
    --lora_dropout 0.05 \
    --lora_target_modules '[gate_proj, down_proj, up_proj]' \
    --train_on_inputs False \
    --add_eos_token False \
    --group_by_length False \
    --prompt_template_name alpaca \
    --lr_scheduler 'cosine' \
    --warmup_steps 100 \
    --wandb_project E4SRec

The infer script is 

torchrun --nproc_per_node=8 --master_port=1234 inference.py \
    --base_model /data06/Llama-2-13b-hf \
    --data_path Beauty \
    --task_type sequential \
    --checkpoint_dir ./LLM4Rec/ \
    --cache_dir cache_dir/ \
    --output_dir ./LLM4Rec/ \
    --batch_size 16 \
    --micro_batch_size 1

The reported results is that

Evaluation for User: 

Precision@1: 8.9433439162903e-05 
 Recall@1: 8.9433439162903e-05 
 MRR@1: 8.9433439162903e-05 
 MAP@1: 8.9433439162903e-05 
 NDCG@1: 8.9433439162903e-05 

Precision@5: 8.9433439162903e-05 
 Recall@5: 0.00044716719581451504 
 MRR@5: 0.0001974988448180775 
 MAP@5: 0.0001974988448180775 
 NDCG@5: 0.0002586554819987668 

Precision@10: 8.943343916290303e-05 
 Recall@10: 0.0008943343916290301 
 MRR@10: 0.00025114116358582666 
 MAP@10: 0.00025114116358582666 
 NDCG@10: 0.0003971278706389283 

Precision@20: 7.825425926754018e-05 
 Recall@20: 0.0015650851853508028 
 MRR@20: 0.0002965062025846168 
 MAP@20: 0.0002965062025846168 
 NDCG@20: 0.0005650891080215324 

Precision@100: 7.154675133032246e-05 
 Recall@100: 0.007154675133032241 
 MRR@100: 0.00040016219874475093 
 MAP@100: 0.00040016219874475093 
 NDCG@100: 0.0015212688144377391
Xu107 commented 6 months ago

The similar issues here.I would like to ask if there are any issues with the evaluation? There are too many decimals,like 0.000.

HestiaSky commented 5 months ago

If you want to use a smaller base model, please use garage-bAInd/Platypus2-13B or garage-bAInd/Platypus2-7B instead. Original LLaMA2 is not instruction-tuned and can not get desirable results.