search

unslothai / unsloth

Finetune Llama 3.2, Mistral, Phi, Qwen 2.5 & Gemma LLMs 2-5x faster with 80% less memory
https://unsloth.ai
Apache License 2.0
18.42k stars 1.29k forks source link

Model folder deleted after saving #1119

Closed giuliabaldini closed 1 month ago

giuliabaldini commented 1 month ago

Hi there,

I have previously saved a model to disk using SFTTrainer.save_model, and I am currently reloading it to save it in different formats. While using unsloth_save_pretrained_gguf, the folder where I was storing my model was deleted, which was unexpected. I guess it makes sense to save space, but I would probably add a flag to make the user control this. The offending line is probably this one.

What do you think, does it make sense? I am happy to write a pull request to add this.

Thank you.

Best, Giulia

danielhanchen commented 1 month ago

Oh yep so for Kaggle and Colab envs we delete the model folder to save space - is this also happening on a local machine?

giuliabaldini commented 1 month ago

It is unfortunately!

giuliabaldini commented 1 month ago

After some investigating I found that this line causes it, I will submit a PR later today!

danielhanchen commented 1 month ago

Appreciate it immensely again!!