Open AnandUgale opened 1 month ago
Did you set the huggingface token in enviroment variable?
Hi @CrasCris,
I have downloaded the Phi-3 model locally and want to access it on a machine that doesn't allow connections to third-party API tokens like Hugging Face. Let's assume I want to access the model on a machine without an internet connection. That's why I haven’t been using the Hugging Face API key. What would you suggest in this situation?
You have to load all the model in the machine and load , i only had made with TrOCR model like this
processor = TrOCRProcessor.from_pretrained('microsoft/trocr-small-handwritten', cache_dir='models/trocr')
or this way
OCR_model = VisionEncoderDecoderModel.from_pretrained('models/trocr')
Checked other resources
Description
Issue
When using the ChatHuggingFace model from the langchain_huggingface library with a locally downloaded model, the system prompts for a Hugging Face API key. This behavior persists even though the model is stored locally and accessed without an internet connection. The issue does not occur when using the llm.invoke() method directly with the HuggingFacePipeline. Description
I have downloaded an LLM model locally and want to access it without an internet connection. The model works correctly when invoked directly using llm.invoke(), but when attempting to use the ChatHuggingFace model, it requires a Hugging Face API key.
Example Code
from transformers import BitsAndBytesConfig from langchain_huggingface import HuggingFacePipeline from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline
Define Quantization
quantization_config = BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype="bfloat16", bnb_4bit_use_double_quant=True )
Initialize LLM and Parameters
model_id = "microsoft/Phi-3-mini-4k-instruct" tokenizer = AutoTokenizer.from_pretrained("../Phi-3-mini-4k-instruct/") model = AutoModelForCausalLM.from_pretrained("../Phi-3-mini-4k-instruct/", quantization_config=quantization_config)
Initialize pipeline
pipe = pipeline( task="text-generation", model=model, tokenizer=tokenizer, max_new_tokens=1024, do_sample=False, repetition_penalty=1.03, )
Initialize HuggingFace Pipeline
llm = HuggingFacePipeline( pipeline=pipe, )
Successful Invocation
llm.invoke("What is HuggingFace?") (Output: "What is HuggingFace?\n\nHuggingFace is a company and an open-source community...")
Using Chat Model
from langchain_huggingface import ChatHuggingFace
llm_engine_hf = ChatHuggingFace(llm=llm) llm_engine_hf.invoke("Hugging Face is")
Error encountered
""" Error Message LocalTokenNotFoundError: Token is required (
token=True
), but no token found. You need to provide a token or be logged in to Hugging Face withhuggingface-cli login
orhuggingface_hub.login
. See https://huggingface.co/settings/tokens """Error Message and Stack Trace (if applicable)
Error Message LocalTokenNotFoundError: Token is required (
token=True
), but no token found. You need to provide a token or be logged in to Hugging Face withhuggingface-cli login
orhuggingface_hub.login
. See https://huggingface.co/settings/tokensSystem Info
linux - ubuntu 20.04.06 LTS Python - 3.8 langchain-huggingface==0.0.3