fix large loss during llama2 post-training

horseee / LLM-Pruner

[NeurIPS 2023] LLM-Pruner: On the Structural Pruning of Large Language Models. Support Llama-3/3.1, Llama-2, LLaMA, BLOOM, Vicuna, Baichuan, TinyLlama, etc.

https://arxiv.org/abs/2305.11627

Apache License 2.0

886 stars 106 forks source link

fix large loss during llama2 post-training #82

Open sidhantls opened 1 month ago

sidhantls commented 1 month ago

Fixes #81

When loading pruned model (output of hf_prune.py) in post_training, cast model to fp32 if base_model is llama2