Lightning-AI / litgpt

20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.
https://lightning.ai
Apache License 2.0
9.77k stars 974 forks source link

Suggest pinned commits in config URLs #1145

Open carmocca opened 6 months ago

carmocca commented 6 months ago

Our tutorials have suggestions like

litgpt finetune lora \
  --config https://raw.githubusercontent.com/Lightning-AI/litgpt/main/config_hub/finetune/llama-2-7b/lora.yaml \
  --lora_r 4

But this has the drawback that it will stop working if:

So we should suggest a specific commit:

https://raw.githubusercontent.com/Lightning-AI/litgpt/396bae5/config_hub/finetune/llama-2-7b/lora.yaml

The disadvantage is that we will need to remember to update this commit. Maybe we could automate this with a GitHub workflow

Thoughts or suggestions?

awaelchli commented 6 months ago

I see two additional options:

  1. The installed litgpt package can know the version it is at and then replace the tag in the URL that the user pastes.
  2. We could support --config litgpt://name.yaml as a way to substitute the URL to the config_hub and automatically choose the right tag in the commit history. This would also make the links shorter for docs.
awaelchli commented 6 months ago

Probably 2 would be better here so users could still use full urls to main if they wanted to deliberately.