Few tweaks in cache paths

[Wauplin]

Hi there! I'm Wauplin, working at HF as huggingface_hub maintainer :hugs: . Very cool project! :fire:

I quickly reviewed the integration with huggingface_hub which looks quite well managed already. I opened this PR to update two things:

• it's preferred to use huggingface_hub.constants.HF_HUB_CACHE rather than hard-coding "~/.cache/huggingface/hub/". First, it will make the project compatible with Windows machine. And moreover it will respect the HF_HOME / HF_HUB_CACHE environment variables if the user chose to locate their files in a different volume. More details about environment variables here.
• when deleting a single file from the cache, you should also remove the associated blob file. Here is a guide on how cache is working in the HF ecosystem. In the current implementation, doing os.remove(".../.../...gguf") is not enough since it would only remove the symlink and not the underlying file (i.e. the "real huge file"). Note that deleting the blob file might break the cache if several revisions of the same repo where downloaded but I don't think it's really relevant in the context here. In general, cache management can be tricky so huggingface-cli delete-cache is the preferred to handle cache properly but I understand the need for a simple API here.

Also a remark about a comment in the code:

Naming doesn't seem to be consistent across 🤗 but it does seem to be consistent across TheBloke's repos, so making some assumptions.

This is definitely true! The Hugging Face Hub is a platform that allow anyone to share their ML artifacts (models, datasets and demos). Artifacts are stored as git repositories in which we don't enforce any structure, given how diverse the ML ecosystem is. It is at the users and libraries discretion to know which models are compatible or not. We still try our best to provide guidelines and curate models metadata (to help with discoverability). So yes, TheBloke's repos are a great starting point for consistency! :smile:

[google-cla[bot]]

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

[bobcatfish]

/gcbrun

[bobcatfish]

Awesome, thank you so much for the fixes and info @Wauplin !! 🙏 🙏 🙏

It looks like in order for us to be able to merge the PR as-is you'll need to sign the Google CLA, let me know if that won't work for you - and no worries if so, I really appreciate the tips and guidance regardless 🥳

[Wauplin]

Hi @bobcatfish, thanks for the quick feedback!

Actually HF signed the Google CLA but I didnot sign my commits correctly. I opened a new PR (https://github.com/GoogleCloudPlatform/localllm/pull/6) that is an exact copy of this one but this time it should be fine. So if it's fine with you, let's close #5 and merge #6 instead :)

[bobcatfish]

sgtm, thanks again @Wauplin !