search

jasonvanf / llama-trl

LLaMA-TRL: Fine-tuning LLaMA with PPO and LoRA
Apache License 2.0
183 stars 23 forks source link

tuning_lm_with_rl.py does not appear to have a file named config.json #5

Open judyhappy opened 1 year ago

judyhappy commented 1 year ago

Hi Jason,

I followed the steps Step 1 - Supervised Fine-tuning, generate "/checkpoints/supervised_llama/" including folders:

checkpoint-2000
checkpoint-3000
checkpoint-4000
final_checkpoint

Step 2 Training Reward Model, generate "/checkpoints/training_reward_model/" including folders:

llama-7b-hf_peft_gpt-4-llm_rm_0_2e-05
peft_last_checkpoint

Step 3 Tuning LM with PPO.

accelerate launch --multi_gpu --num_machines 1  --num_processes 2     tuning_lm_with_rl.py     --log_with wandb     --model_name ./checkpoints/supervised_llama/     --reward_model_name ./checkpoints/training_reward_model/     --adafactor False     --tokenizer_name ./data/model/     --save_freq 100     --output_max_length 128     --batch_size 8     --gradient_accumulation_steps 8     --batched_gen True     --ppo_epochs 4     --learning_rate 1.4e-5     --early_stopping True     --output_dir './checkpoints/tuning_llama_rl/'

But there is an Error:

CUDA SETUP: Loading binary /usr/local/lib/python3.8/dist-packages/bitsandbytes/libbitsandbytes_cuda117_nocublaslt.so...
Traceback (most recent call last):
  File "tuning_lm_with_rl.py", line 159, in <module>
    tokenizer = AutoTokenizer.from_pretrained(script_args.model_name)
  File "/usr/local/lib/python3.8/dist-packages/transformers/models/auto/tokenization_auto.py", line 657, in from_pretrained
    config = AutoConfig.from_pretrained(
  File "/usr/local/lib/python3.8/dist-packages/transformers/models/auto/configuration_auto.py", line 916, in from_pretrained
    config_dict, unused_kwargs = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/transformers/configuration_utils.py", line 573, in get_config_dict
    config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/transformers/configuration_utils.py", line 628, in _get_config_dict
    resolved_config_file = cached_file(
  File "/usr/local/lib/python3.8/dist-packages/transformers/utils/hub.py", line 380, in cached_file
    raise EnvironmentError(
OSError: ./checkpoints/supervised_llama does not appear to have a file named config.json. Checkout 'https://huggingface.co/./checkpoints/supervised_llama/None' for available files.

There is no config.json under supervised_llama or training_reward_model.

judyhappy commented 1 year ago

Step 1 and Step 2 don't generate any config.json. So which config.json should be used for step 3?

43zxj commented 1 year ago

Step 1 and Step 2 don't generate any config.json. So which config.json should be used for step 3?

Hello, have you solved this problem yet? Could you tell me which config.json should be used for step 3? Thank you!!!!