search

princeton-nlp / LLM-Shearing

[ICLR 2024] Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning
https://arxiv.org/abs/2310.06694
MIT License
539 stars 41 forks source link

Can you provide script without using slurm or sbatch? #6

Closed hanlinxuy closed 10 months ago

hanlinxuy commented 10 months ago

I would like to do some experiments with llm-shearing but I do not have a cluster with SLURM. (I only have some GPUs). It would be nice if you guys can provide some scripts that can be used without sbatch.

xiamengzhou commented 10 months ago

Hi! I updated the scripts (pruning.sh and continue_pretrain.sh) to include commands to run the scripts directly. Essentially you only need to replace the sbatch head with composer train.py. Let me know if you need additional help.

hanlinxuy commented 10 months ago

Hi! I updated the scripts (pruning.sh and continue_pretrain.sh) to include commands to run the scripts directly. Essentially you only need to replace the sbatch head with composer train.py. Let me know if you need additional help.

Thank you very much! I am going to test them!