search

neuralmagic / sparsezoo

Neural network model repository for highly sparse and sparse-quantized models with matching sparsification recipes
Apache License 2.0
371 stars 25 forks source link

Reproduction problem #481

Closed H-cool closed 7 months ago

H-cool commented 7 months ago

Hello, I am reproducing the llama2-7b-gsm8k_llama2_pretrain-pruned60_quantized model. When fine-tuning the llama2 model without pruning, do you use the file https://github.com/IST-DASLab/SparseFinetuning/blob/main/scripts/train/train.py to fine-tune, specifically? How are the parameters selected? Thank you very much for your answer.

mgoin commented 7 months ago

Hey @H-cool we have a guide for reproducing Llama 2 7B GSM8k sparse finetuning results using SparseML, please check it out here! https://docs.neuralmagic.com/llms/guides/sparse-finetuning-llm-gsm8k-with-sparseml