unslothai / unsloth

Finetune Llama 3.2, Mistral, Phi, Qwen 2.5 & Gemma LLMs 2-5x faster with 80% less memory
https://unsloth.ai
Apache License 2.0
18.37k stars 1.28k forks source link

Feat/kaggle-gguf-on-tmp #1307

Open Erland366 opened 2 days ago

Erland366 commented 2 days ago

Now we move all GGUF Process to /tmp so user can convert GGUF using Kaggle

We manage to save the GGUF on /tmp but now the problem is how user will download the GGUF since you can't access /tmp through GUI and FileLink seems not working .-.

Also fixing https://github.com/unslothai/unsloth/issues/1222 since HFFileSystem is hardcoded to use linux format of forward slash instead of adapting to the user's OS

Here's my run for GGUF on Mistral 7b (I already fix the wrong name on the print, I am to lazy to rerun it again LOL)

image

Because the save_pretrained_gguf still not "solved" (need to discuss how to approach the download process). I put it as draft first