stephanj
commented
1 month ago Here are some suggestions to work around this issue:
-
Manual Download and Placement: You can manually download the model files from Hugging Face when you have access to a network that can reach huggingface.co. Then, you can transfer these files to your server. The typical directory structure for pretrained models is:
~/.cache/huggingface/hub/
Within this directory, create a folder structure that mirrors the model's name on Hugging Face. For example:
~/.cache/huggingface/hub/models--llama3-8b-sfr/
Place the downloaded files (like config.json, model.safetensors, tokenizer.json, etc.) in this directory.
-
Use Offline Mode: If you've manually placed the files as described above, you can use Transformers' offline mode. Set the following environment variable before running your script:
export TRANSFORMERS_OFFLINE=1This tells the library to only look for local files and not try to download anything.
-
Local Model: If you have the model files in a local directory, you can specify the path directly instead of using the model ID:
from transformers import AutoTokenizer tokenizer = AutoTokenizer.from_pretrained("/path/to/your/local/model/directory") -
Use a Mirror: If your server can access other domains, you might be able to use a Hugging Face mirror. You can specify a custom endpoint when loading the model:
from transformers import AutoTokenizer tokenizer = AutoTokenizer.from_pretrained("llama3-8b-sfr", use_auth_token=True, endpoint="https://your-mirror-url.com") -
Proxy Configuration: If your server requires a proxy to access external resources, you can configure it:
import os os.environ['HTTP_PROXY'] = 'http://your-proxy:your-port' os.environ['HTTPS_PROXY'] = 'https://your-proxy:your-port'Set these before running your script.
-
Network Troubleshooting: Ensure that your server's firewall isn't blocking outgoing connections to huggingface.co. You might need to whitelist this domain in your network settings.
artistlu
I'm encountering an issue when trying to use the Exor project. It appears that my server is unable to access the
huggingface.codomain, which is preventing me from downloading the required model files.Is there a way for me to manually download the model files and then place them in the appropriate directory on my server? If so, could you please provide the steps or the specific directory path where I should put the downloaded model files?
Alternatively, do you have any other suggestions on how I can work around this issue and successfully use the Exo on my server?