Add support for branches

[minipasila]

So it turns out the model I want to use to finetune for some reason doesn't have the model files in the main branch.. but it has bunch of the checkpoints uploaded in different branches like "1000B" but there doesn't seem to be anyway for me to select that branch in Unsloth. If you could specify what branch to use that would be nice.

[danielhanchen]

Weird - how would one do it in normal HF?

[minipasila]

I know Ooba can do it just fine https://github.com/oobabooga/text-generation-webui/blob/abe5ddc8833206381c43b002e95788d4cca0893a/download-model.py#L53 (in Ooba you just add the branch with user/model:branchname)

[erwe324]

Hmm, interesting scenario. Try this.

1. Download the model yourself on the system you have unsloth installed. Downloading can be manually done or via git-lfs clone https://huggingface.co/LumiOpen/Viking-33B/tree/1000B

2. git-lfs is required to download files stores via lfs.

Assuming you have downloaded the model in /home/user/Viking-33B pass the name of the folder as follows:

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = "/home/user/Viking-33B", #your model location.
    max_seq_length = max_seq_length,
    dtype = dtype,
    load_in_4bit = load_in_4bit,
    # token = "hf_...", # use one if using gated models like meta-llama/Llama-2-7b-hf
)

And this should work.

[minipasila]

That would be useful if I weren't using this on Runpod/Colab. Though my solution was similar in that I downloaded the model and then reuploaded it back to Huggingface... and then used that instead.

[erwe324]

Excellent approach. It never crossed my mind