Wandb course - training and fine tuning LLM

[manisnesan]

Notes - Evernote

• LLM Finetuning Code by Ayush & Quick Starter Guide https://github.com/Lightning-AI/lit-gpt/blob/main/tutorials/neurips_challenge_quickstart.md
• Starter Kit https://lightning.ai/pages/community/tutorial/neurips2023-llm-efficiency-guide/ by Sebastian Raschka
• NeurIPS Large Language Model Efficiency Challenge: 1 LLM + 1GPU + 1Day.
• NeurIPS LLM Challenge page
• NeurIPS LLM Challenge Discord

Progress

• [x] Foundations
• [ ] Evaluation

[manisnesan]

https://github.com/brevdev/notebooks/blob/main/mistral-finetune.ipynb

Fine tune mistral 7B using Qlora