search

AGI-Edgerunners / LLM-Adapters

Code for our EMNLP 2023 Paper: "LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Models"
https://arxiv.org/abs/2304.01933
Apache License 2.0
1.02k stars 92 forks source link

support ChatGLM #8

Closed HZQ950419 closed 1 year ago

HZQ950419 commented 1 year ago

ChatGLM is supported by this PR.

CUDA_VISIBLE_DEVICES=1 python finetune.py \ --base_model 'THUDM/chatglm-6b' \ --data_path 'math_data.json' \ --output_dir './trained_models/chatglm_test' \ --batch_size 16 \ --micro_batch_size 4 \ --num_epochs 3 \ --learning_rate 3e-4 \ --cutoff_len 256 \ --val_set_size 120 \ --adapter_name lora \ --use_gradient_checkpointing

This should work fine for fintuning.