jiaweizzhao / GaLore

GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection
Apache License 2.0
1.24k stars 131 forks source link

Release of Trained Models #38

Open JLake310 opened 2 months ago

JLake310 commented 2 months ago

Hi, thanks very much for sharing your impressive work!

Would it be possible to release the trained model (e.g., using the script below)? It would greatly facilitate reproducibility efforts. Thank you for considering this request. https://github.com/jiaweizzhao/GaLore/blob/1b36c33782bdd74a4d6a4f51bc626ef67f51011f/scripts/benchmark_c4/llama_7b.sh