Unsloth should probably not alter or delete source models.

[rwl4]

I always keep a local copy of every model I work with regularly in a models directory on my server. I noticed that unsloth will write input_embeddings.pt and output_embeddings.pt to the source model directory, then in some cases will delete the source. This means that I now have to re-download the model. This is far from ideal.

I've tried in vain to do various workarounds such as:

• making the directory read-only (breaks unsloth due to permissions, obviously)
• making the directory and base model contents owned by root but the directory having 1777 permissions (all users get rwx permissions + sticky, same as your standard /tmp directory, but unsloth wants to modify the tokenizer)

But those tricks always fail.

Can you instead create a different directory under huggingface's cache directory instead? For example, maybe for Mistral-7B-v0.3 you might create a tmp directory called ~/.cache/unsloth/path-to-using-dashes-instead-of-slashes-Mistral-7B-v0.3-unsloth-work and write your diffs there. You could even be really fancy and make it attempt a symlink to the original files, then write your updated files to that directory in lieu of the symlinked version.

I'm getting to the point where I will have to switch to using ZFS copy-on-write or do an overlay mount or similar since I literally keep having to place the original model back in place on an almost daily basis

[danielhanchen]

Hmm interesting point and good idea - hmm weird it's deleting the entire folder? I'll check why this is happening - maybe my caching methods are not correct