fix: update cache parameters to enable local model storage

[0xlws]

• updated cache parameters in loaders to reach model downloader
• added custom demucs script to allow configuring cache dir
• added /models/ to .gitignore to ignore downloaded models fixes: #234, #218, #100

usage:

from audiocraft.models import MusicGen
from audiocraft.models import MultiBandDiffusion
import torch
USE_DIFFUSION_DECODER = False
CACHE_DIR = 'models'

# Using small model, better results would be obtained with `medium` or `large`.
model = MusicGen.get_pretrained('facebook/musicgen-small', cache_dir=CACHE_DIR)
if USE_DIFFUSION_DECODER:
    mbd = MultiBandDiffusion.get_mbd_musicgen(cache_dir=CACHE_DIR)

[nel1991]

usage:

from audiocraft.models import MusicGen from audiocraft.models import MultiBandDiffusion import torch USE_DIFFUSION_DECODER = False CACHE_DIR = 'models'

Using small model, better results would be obtained with medium or large.

model = MusicGen.get_pretrained('facebook/musicgen-small', cache_dir=CACHE_DIR) if USE_DIFFUSION_DECODER: mbd = MultiBandDiffusion.get_mbd_musicgen(cache_dir=CACHE_DIR)

@0xlws where is this located on the audiocraft? what folder, and whats the name of the file? do we need to replace a specific line there? or just add that?

[JadeCopet]

Thanks. We would rather not add more complex behavior in the model API for now and these cache dir location can be controlled by environment variables:

• The AudioCraft models can be downloaded at a custom location specifying the AUDIOCRAFT_CACHE_DIR as mentioned in the README.
• For Demucs the download location can be controlled as any other model available through the torch hub, setting the TORCH_HOME variable.
• Finally, for the other models imported from Hugging Face transformers library, you should refer to their documentation.

[0xlws]

@JadeCopet @adefossez big Thanks for this amazing repo.

i just now tried using the global variable (i wasnt handy with Python env vars), and it works for the musicgen model, but again the t5 lm still goes into the .cache.

in point 3 you mention 'other models', not sure but i guess you were referring to that t5 lm. i went over the huggingface page and could not find anything clear related...

the way it was solved was by 'simply' passing in the cache_dir variable.

instead of passing it in at the root call we can also use the env var instead as such before these lines:

   correction:
    # if cache_dir is None: 
    cache_dir = os.environ.get('AUDIOCRAFT_CACHE_DIR', None)

and pass it as such. this way the language model gets stored in the same specified folder.

id love to hear your thoughts

[nel1991]

is this related to my problem @0xlws ?

[nel1991]

@0xlws where is this code located on the folder ? if cache_dir is None: cache_dir = os.environ.get('AUDIOCRAFT_CACHE_DIR', None)

[nel1991]

please help

[0xlws]

@nel1991

is this related to my problem @0xlws ?

yes but youre experiencing issues even after following the same instructions i used, which work in my case and still give errors in your case.

@0xlws where is this code located on the folder ? if cache_dir is None: cache_dir = os.environ.get('AUDIOCRAFT_CACHE_DIR', None)

as @JadeCopet mentioned the use of the env vars, i took the existing implementation from: https://github.com/facebookresearch/audiocraft/blob/5ecdc68ef7a442ed81b6d316fc5029b5a2c16290/audiocraft/models/loaders.py#L44

actual function being on line 34.

so i would also advice you to use the env vars. in the collab cell i ran:

import os

os.environ['AUDIOCRAFT_CACHE_DIR'] = 'models'
os.environ['TORCH_HOME'] = 'models'

which also created the models folder and stored the models, t5 model was still being written inside the .cache folder. so in your case this may not be solved as it will still try to write to your unwritable .cache folder. thats why i still propose to set the cache for the model in the function using the same env var instead.

[nel1991]

under model/loaders.py under demos\musicgen_app.py is the code correct on the top?

im still getting an error:

[0xlws]

@nel1991 i notice youre still using my repo, i may have opened a can of worms for you. i just updated my repo to align again with the main repo, and using the env vars.

---

edit: added a commit to support storing T5.

---

i tested it and it works the same. please make sure to update the latest changes, the ROOT = Path... ... (on lines 31 - 36) will be removed if updated.

[nel1991]

i did download the main folder and the error is unable to find the cache folder, when i downloaded yours, your updated repo, still the same error. your repository is already updated right? once i downloaded your repository again, it will show the changes. But the error still the same. i dont know what to do