search

RUC-NLPIR / FlashRAG

⚡FlashRAG: A Python Toolkit for Efficient RAG Research
https://arxiv.org/abs/2405.13576
MIT License
933 stars 72 forks source link

gated repo (model) #51

Closed codecodebear closed 2 days ago

codecodebear commented 1 week ago

What are ways (interfaces) where config is able to directly connecting with HuggingFace such that we don't need to download models or the code can automatically download models for us? For example, if I just use 'model2path': {'e5': 'intfloat/e5-base-v2', 'llama3-8B-instruct': 'meta-llama/Meta-Llama-3-8B-Instruct'}, in config_dict, I get the following error for gated repo:

OSError: You are trying to access a gated repo.
Make sure to have access to it at https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct.

Right now, I have to manually download models and point to its directory when running in Google Colab, which is inconvenience.

ignorejjj commented 1 week ago

Our library supports direct connection with huggingface. But to download models that require permissions, you need to login first.

from huggingface_hub import login
login()