kohya-ss / sd-scripts

Apache License 2.0
5.12k stars 852 forks source link

How to change HuggingFace directory #384

Open mbastias opened 1 year ago

mbastias commented 1 year ago

I'm sorry to use an issue for this. My C: drive is very full and the hugginface cache is using about 30Gb so I want to move it to one of my bigger drives. On SD webui I managed to do it by doing a set for the HF_HOME, I thought that this setting was going to carry over to Kohya but it did not, as it downloaded the cached files again.

Is it possible to change the directory? And if it is, how?

kohya-ss commented 1 year ago

I'm not sure about caching, but according to the next issue. setting environmental variable HF_HOME will solve it: https://github.com/huggingface/transformers/issues/8703

mbastias commented 1 year ago

Where do I set it? on the venv? Sorry I'm very new this environment.

sdake commented 1 year ago

@mbastias if you are running bash, which most Linux environments use by default, in your shell:

export HF_HOME=${HOME}/cache

A better option is to add this to the default startup. To do this, add export HF_HOME=${HOME}/cache to the ${HOME}/.profile file.

For more detail, reference the .profile man page.

rezponze commented 1 year ago

I'm on Windows and I have the same problem. My C drive is full and I would like to move the .cache/huggingface folder to another drive.

mbastias commented 1 year ago

I'm on Windows and I have the same problem. My C drive is full and I would like to move the .cache/huggingface folder to another drive.

I seem to have solve it adding "set HF_HOME=D:\SDCache" like this:

@echo off

:: Activate the virtual environment
call .\venv\Scripts\activate.bat

:: Validate the requirements and store the exit code
python.exe .\tools\validate_requirements.py

set HF_HOME=D:\SDCache

:: If the exit code is 0, run the kohya_gui.py script with the command-line arguments
if %errorlevel% equ 0 (
    python.exe kohya_gui.py %*
)

Replace D:\SDCache with the path you want

aearone commented 1 year ago

I'm on Windows and I have the same problem. My C drive is full and I would like to move the .cache/huggingface folder to another drive.

I seem to have solve it adding "set HF_HOME=D:\SDCache" like this:

@echo off

:: Activate the virtual environment
call .\venv\Scripts\activate.bat

:: Validate the requirements and store the exit code
python.exe .\tools\validate_requirements.py

set HF_HOME=D:\SDCache

:: If the exit code is 0, run the kohya_gui.py script with the command-line arguments
if %errorlevel% equ 0 (
    python.exe kohya_gui.py %*
)

Replace D:\SDCache with the path you want

You are the best! Thank you! I wrote the same line in setup.bat to install the acceleration files in a different folder. It really works. And hopefully it won't break anything, huh:)

sdake commented 1 year ago

nice work @aearone !

Thank you for sharing your work, -steve

sdake commented 1 year ago

rather thank you @mbastias for solving this problem for Windows.

thank you, -steve

mylesgoose commented 7 months ago

that seems hard. i just type env down in the search box in windows 11. and that opens the environmental variables. i click new. and add this variable name HF_HOME variable value H:\Cache then okay. and it seems to be persistent.

SinoCHU commented 6 months ago

$Env:HF_HOME = "F:\HF_Cache" 将 F:\HF_Cache 替换为您想要的路径 powershell里面执行