Closed leoozy closed 1 month ago
Save pytorch_model.bin at the output file
There is only model_adapter.bin. No model saved
command = f"/opt/anaconda3/envs/open-sora/bin/accelerate launch -m axolotl.cli.train {args.yaml_path} --deepspeed {args.deepspeed}"
base_model: /zhangjunlei/download/models/Meta-Llama-3-8B-Instruct model_type: LlamaForCausalLM tokenizer_type: AutoTokenizer load_in_8bit: false load_in_4bit: false strict: false datasets: - path: /zhangjunlei/download/data/webagent/axo_vwa_cot_json_v3.json ds_type: json type: alpaca dataset_prepared_path: val_set_size: 0.05 output_dir: /zhangjunlei/code/axolotl/output/llama3.1_70b_lora_vwa_0801_1_debug3 sequence_len: 8192 sample_packing: true pad_to_sequence_len: true adapter: lora lora_model_dir: lora_r: 512 lora_alpha: 1024 lora_dropout: 0.05 lora_target_linear: true lora_fan_in_fan_out: wandb_project: wandb_entity: wandb_watch: wandb_name: wandb_log_model: gradient_accumulation_steps: 2 micro_batch_size: 1 num_epochs: 4 optimizer: adamw_bnb_8bit lr_scheduler: cosine learning_rate: 0.0002 train_on_inputs: false group_by_length: false bf16: auto fp16: tf32: false gradient_checkpointing: true early_stopping_patience: resume_from_checkpoint: local_rank: logging_steps: 1 xformers_attention: flash_attention: true s2_attention: warmup_steps: 10 evals_per_epoch: 4 eval_table_size: eval_max_new_tokens: 128 saves_per_epoch: 8 debug: deepspeed: weight_decay: 0.0 fsdp: fsdp_config: special_tokens: eos_token: "<|eot_id|>" pad_token: "<|end_of_text|>" eval_sample_packing: False
No response
3.10
main
Please check that this issue hasn't been reported before.
Expected Behavior
Save pytorch_model.bin at the output file
Current behaviour
There is only model_adapter.bin. No model saved
Steps to reproduce
Config yaml
Possible solution
No response
Which Operating Systems are you using?
Python Version
3.10
axolotl branch-commit
main
Acknowledgements