[FEATURE REQUEST] (Offering to help) - Unified model management

[d8ahazard]

Hey there @gitmylo.

Just wanted to check - would you be opposed to me adding a "unified" model management system? Basically, I'd just add a custom method to download models to one central location, then use those instead of from HF Hub cache, which is a nightmare to manage if sharing a folder amongst multiple installs, etc.

I can start working on a PR in the next few days, just wanted to check with you first...

[gitmylo]

As a replacement for the current environment overrides in main.py? What would be the difference between this and adding a flag to set the model base dir? (instead of just having a toggle to revert to default) Currently it uses os.path.join(os.path.dirname(os.path.realpath(__file__))/models as the base path, unless the --no-data-cache flag is set, in which case it uses the default cache path for huggingface hub etc.

Is your plan to let users select a different path than the current model data path, or something different, like managing them a whole other way?

[d8ahazard]

As a replacement for the current environment overrides in main.py? What would be the difference between this and adding a flag to set the model base dir? (instead of just having a toggle to revert to default) Currently it uses os.path.join(os.path.dirname(os.path.realpath(__file__))/models as the base path, unless the --no-data-cache flag is set, in which case it uses the default cache path for huggingface hub etc.

Is your plan to let users select a different path than the current model data path, or something different, like managing them a whole other way?

In other projects where I've encountered this issue, I basically create a method called "get_model_path" or something, which takes a HF model URL and and downloads it to a specified folder - which would still be the /data/models folder of the project - then returns the path.

But, I'd specifically find the bits that are still downloading to the hf_cache folder, and remove the dependency on the .locks folder and stuff now, which is an issue in my implementation.

[gitmylo]

That sounds great, actually. I would really appreciate that.

[d8ahazard]

That sounds great, actually. I would really appreciate that.

QQ - can you point me to the spot in the code where the suno/bark model is actually downloaded/loaded? That one's a bit tricky, I think. :D

[gitmylo]

The call to the original download function can be found here (at webui/modules/implementations/patches/bark_generation.py line 563)

[d8ahazard]

Excellent, thank you.

So, the round-up process is going well. So far, I've got like 13 or so hooks added to use the same model manager:

It supports both url downloads, and hf hub downloads, and for single files and whole repos, including filtering and stuff for the various filenames.

Not only do I have the majority of the "big" model downloads now under our control, but I also added a bit that logs every download, so we can add a "download all the things" flag or something which fetches everything. Then all the models can just be there if a person desires. ;)

Also, updated the bit that lists the whisper models to always contain the full list, including the new v3 model.

Still doing some cleanup and stuff as I have time, will submit a PR when done. It will likely be a messy PR, so I'll apologize beforehand. I'll try to keep the commits clean, but I did remove a few things (like the hubert manager), and do some code cleanup as well.

Question - where is "whatever" that goes into audio-to-audio triggered to download?

And, where would you like the encodec and google-bert models to live?

[gitmylo]

Question - where is "whatever" that goes into audio-to-audio triggered to download?

I'm not sure what you mean by this. Can you elaborate?

And, where would you like the encodec and google-bert models to live?

encodec and google-bert can just have their own folder, like you have them in the screenshot above.