facebookresearch / audiocraft

Audiocraft is a library for audio processing and generation with deep learning. It features the state-of-the-art EnCodec audio compressor / tokenizer, along with MusicGen, a simple and controllable music generation LM with textual and melodic conditioning.
MIT License
20.5k stars 2.06k forks source link

There was a problem when trying to write in your cache folder (C:\Users\Lenovo\.cache). You should set the environment variable TRANSFORMERS_CACHE to a writable directory. #246

Open nel1991 opened 1 year ago

nel1991 commented 1 year ago
transformererror

I have problems in running the audiocraft. Every time i run the command, python demos\musicgen_app.py, it says that i need to fix the cache. can someone help me?

The error is: There was a problem when trying to write in your cache folder (C:\Users\Lenovo.cache). You should set the environment variable TRANSFORMERS_CACHE to a writable directory.

0xlws commented 1 year ago

hey @nel1991 , i made a pr(#239) to enable passing a cache dir. it seems like you cant write to .cache, so setting a different folder may help there, otherwise check the permissions in that folder (i use mac so i may not be helpful for windows specific issues).

nel1991 commented 1 year ago

usage:

from audiocraft.models import MusicGen from audiocraft.models import MultiBandDiffusion import torch USE_DIFFUSION_DECODER = False CACHE_DIR = 'models'

Using small model, better results would be obtained with medium or large. model = MusicGen.get_pretrained('facebook/musicgen-small', cache_dir=CACHE_DIR) if USE_DIFFUSION_DECODER: mbd = MultiBandDiffusion.get_mbd_musicgen(cache_dir=CACHE_DIR)

@0xlws where is this located on the audiocraft? what folder, and whats the name of the file? do we need to replace a specific line there? or just add that?

0xlws commented 1 year ago

its still a pull request, currently these modifications are in my fork: https://github.com/0xlws/audiocraft. honestly i don't think there is a user friendly way to change it yet because of the missing parameter, thats why when its added its just 1 parameter away.

there were multiple places where the missing parameter had to be added to reach the function that downloads the models. you can see the changes: https://github.com/facebookresearch/audiocraft/pull/239/files.

as i dont use the gradio-ui, i havent updated that file, but it could be passed as:

def load_model(version='facebook/musicgen-melody', cache_dir='your_models_dir' ): # added here
    global MODEL
    print("Loading model", version)
    if MODEL is None or MODEL.name != version:
        MODEL = MusicGen.get_pretrained(version, cache_dir=cache_dir) # and passed on here

or better yet through the ui in a field

(the above would still only work once the cache_dir parameter is passed all the way down)


edit: and this was the specific file i was referring to: https://github.com/0xlws/audiocraft/blob/main/demos/musicgen_demo.ipynb

nel1991 commented 1 year ago

Theres still an error:

FileNotFoundError: [WinError 3] The system cannot find the path specified: 'C:\Users\Lenovo\.cache\models--t5-base'

What i did, i created another folder, and did all the installation, but when i run it, thats error i got

nel1991 commented 1 year ago

@0xlws Theres still an error:

FileNotFoundError: [WinError 3] The system cannot find the path specified: 'C:\Users\Lenovo.cache\models--t5-base'

What i did, i created another folder, and did all the installation, but when i run it, thats error i got

0xlws commented 1 year ago

@nel1991 im not really sure exactly what steps you went trough so its hard to tell what you should do next... if the problem is related to the folder not existing it could be necessary to create it first. either modify the script or make sure you create it manually.

this is how i handled it in the script:

from pathlib import Path
import os

ROOT = Path(__file__).parent.parent  # my script is in '/local_scripts/example.py' same level as '/demos/demo.py'

print(ROOT)

OUTPUTS_PATH = ROOT / "outputs"
MODELS_PATH = ROOT / "models"

# make sure they exist
os.makedirs(OUTPUTS_PATH, exist_ok=True)
os.makedirs(MODELS_PATH, exist_ok=True)
def load_model(version='facebook/musicgen-melody', cache_dir=MODELS_PATH ): # added here
    global MODEL
    print("Loading model", version)

    if MODEL is None or MODEL.name != version:
        MODEL = MusicGen.get_pretrained(version, cache_dir=cache_dir) # and passed on here

let me know if that helps

nel1991 commented 1 year ago

to tell you honestly im new to python. when you say "either modify the script" where is the script located in the audiocraft folder? do i have to create a separate file for the script? the code that you shown on the top? do i have to create that? or is it on the folder on the audiocraft? if its on the audiocraft, whats the name of the file? please help

nel1991 commented 1 year ago

@0xlws to tell you honestly im new to python. when you say "either modify the script" where is the script located in the audiocraft folder? do i have to create a separate file for the script? the code that you shown on the top? do i have to create that? or is it on the folder on the audiocraft? if its on the audiocraft, whats the name of the file? please help

nel1991 commented 1 year ago

@0xlws to tell you honestly im new to python. when you say "either modify the script" where is the script located in the audiocraft folder? do i have to create a separate file for the script? the code that you shown on the top? do i have to create that? or is it on the folder on the audiocraft? if its on the audiocraft, whats the name of the file? please help

Is that code you shown above, located in this audiocraft folder? image

0xlws commented 1 year ago

@nel1991 i think youre sending multiple messages on accident.

i was assuming u used the gradio ui, the load_model function is inside demos folder, musicgen_app.py line 90. also asumming you cloned https://github.com/0xlws/audiocraft, otherwise it would not work or you would have to make a few changes in the files mentioned.

you could replace the load_model function starting on line 90 with:

from pathlib import Path
import os

def load_model(version='facebook/musicgen-melody'): 
    global MODEL
    print("Loading model", version)

    root_path = Path(__file__).parent.parent
    print("Root path: ", root_path) 

    models_path = root_path / "models"
    os.makedirs(models_path, exist_ok=True)

    if MODEL is None or MODEL.name != version:
        MODEL = MusicGen.get_pretrained(version, cache_dir=models_path)

that is the function that loads the model using the gradio ui, i added the folder creation in the function for 'simplicity'.

using 'Path and file' i dont think is the most beginner friendly syntax, but its getting the path the file is in and going up to its parents to get the root project dir so from there i create the models folder to store the models in.

hope that helps and its not too confusing

nel1991 commented 1 year ago

@0xlws image i applied your codes, i got that error, "TypeError: get_pretrained() got an unexpected keyword argument 'cache_dir'", what does it mean?

0xlws commented 1 year ago

@nel1991 im making a bunch of assumptions here, but it looks like you didnt cloned my fork which has the keyword argument added: https://github.com/0xlws/audiocraft/blob/30ed350da0bf0ead625dcd5e98ce9f02e94f5685/audiocraft/models/musicgen.py#L88

either fetch my fork or clone it in a different folder then that should work

nel1991 commented 1 year ago

wait, what i did is i just clone this "https://github.com/0xlws/audiocraft.git" cause i thought everything is already updated on that repository

0xlws commented 1 year ago

i made it a draft because i noticed how maybe the demucs part is a bit much, and also i only tried it on musicgen_demo.ipynb to conclude its working, overlooking the other 2 scripts.

the passing of the parameter should not give that error in that repo, please double check if your files contain the updated scripts by checking if the functions contain the keyword argument cache_dir and make sure to restart and reload so it reads any changes if thats an issue.

nel1991 commented 1 year ago

I've doubled checked and restart, i still have that error: image

image

nel1991 commented 1 year ago

ive doubled checked the files, the changes that youve made on the repository is just the same, still i got the same error.

0xlws commented 1 year ago

i see, ive updated my repo and just added the changes shown here to the script so you dont have to do it. its not how i would leave it but it should work, please fetch the update and let me know if that works

nel1991 commented 1 year ago

image I still got the same error, i actually updated the files on you mention. still the same:

image Ive already made some changes, i still have the same error

0xlws commented 1 year ago

@nel1991 notice the difference in commit, i think youre still looking at the previous commit?

Screenshot 2023-08-24 at 18 16 00
nel1991 commented 1 year ago

image

I recieved this message from you, when you click the link, it will go here:

image

0xlws commented 1 year ago

if you want please show the screenshot or code snippet of the get_pretrained function inside MusicGen class inside audiocraft/models/musicgen.py.

it should look like: https://github.com/0xlws/audiocraft/blob/517967ebdae878086ec32521a8b182bf5af2a810/audiocraft/models/musicgen.py#L88

it should expect the keyword argument

nel1991 commented 1 year ago

image image image image image image image image image image image

thats the entire code.

0xlws commented 1 year ago

thanks, the part in question looks correct... i just tried it myself locally and its downloading as usual:

Screenshot 2023-08-25 at 08 21 03 Screenshot 2023-08-25 at 08 21 22

i recommend you do a clean install.... here is what i did:

  1. Clone the repository into a new folder, using the command: git clone https://github.com/0xlws/audiocraft.git.
  2. Pip3 install torch using the command: pip3 install torch. # i had to instal torch first
  3. Then install all requirements using pip3 install -r requirements.txt.
  4. Run the following code in VSCode terminal: python -m demos.musicgen_app --share.
  5. Open the Gradio UI.
  6. Input your prompt, select a short duration, set the model to "small" and click "submit".

i did get an error i did not get before which was protobuf not being installed (normally i use colab), so i added it to the requirements.txt, installed, and it worked fine

let me know if it helps

nel1991 commented 1 year ago

I still have an error: when you run the command: python -m demos.musicgen_app --share. thats the error i got, so this is not the right command image

When i enter this command: python demos\musicgen_app.py image

But still i have this error: Im using the annaconda prompt image

image

0xlws commented 1 year ago

a bunch of things going on, but mainly it may be because in the first picture you added a period at the end. its: python -m demos.musicgen_app --share not: python -m demos.musicgen_app --share.

please make sure to check for typos in your commands.


other then that i notice youre not using a local venv, that may cause some troubles later on. i recommend you use a local python env, heres how i do it, this assumes it cloning into the default 'audiocraft' folder:

git clone https://github.com/0xlws/audiocraft.git
cd audiocraft
# create venv
python3 -m venv venv 
# activate local venv
source bin/activate/venv 
pip3 install torch
pip3 install -r requirements.txt

etc. gradio ui: python -m demos.musicgen_app --share

nel1991 commented 1 year ago

I still have the same error, i followed that. the error really is the cache_dir, image is empty

nel1991 commented 1 year ago

even if i re install its still the same

0xlws commented 1 year ago

I still have the same error, i followed that. the error really is the cache_dir, image is empty

i made a mistake in my repo where i merged over the main, i corrected it now.

if you started in a clean/empty environment and it still does not store the models correctly, im afraid there may be other problems i cant pinpoint exactly the cause of, it may be operating system related

nel1991 commented 10 months ago

im still stuck on this problem, the python that you use, is it 3.9 or 3.10? does it matter with the version of python? Because i saw in youtube, he did use the 3.10

nel1991 commented 10 months ago

@0xlws im still stuck on this problem, the python that you use, is it 3.9 or 3.10? does it matter with the version of python? Because i saw in youtube, he did use the 3.10