Bad request: The following `model_kwargs` are not used by the model: ['return_full_text', 'stop', 'watermark', 'stop_sequences'] (note: typos in the generate arguments will also show up in this list)

michaelCHU95 commented 6 months ago

Checked other resources

[X] I added a very descriptive title to this issue.
[X] I searched the LangChain documentation with the integrated search.
[X] I used the GitHub search to find a similar question and didn't find it.
[X] I am sure that this is a bug in LangChain rather than my code.
[X] The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

from langchain.prompts import PromptTemplate from langchain_community.llms.huggingface_endpoint import HuggingFaceEndpoint from langchain.chains import LLMChain

llm = HuggingFaceEndpoint( repo_id="google/flan-t5-large", temperature=0, max_new_tokens=250, huggingfacehub_api_token=HUGGINGFACE_TOKEN )

prompt_tpl = PromptTemplate( template="What is the good name for a company that makes {product}", input_variables=["product"] )

chain = LLMChain(llm=llm, prompt=prompt_tpl) print(chain.invoke("colorful socks"))

Error Message and Stack Trace (if applicable)

Traceback (most recent call last): File "/Users/michaelchu/Documents/agent/agent.py", line 20, in print(chain.invoke("colorful socks")) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/michaelchu/Documents/agent/venv/lib/python3.12/site-packages/langchain/chains/base.py", line 163, in invoke raise e File "/Users/michaelchu/Documents/agent/venv/lib/python3.12/site-packages/langchain/chains/base.py", line 153, in invoke self._call(inputs, run_manager=run_manager) File "/Users/michaelchu/Documents/agent/venv/lib/python3.12/site-packages/langchain/chains/llm.py", line 103, in _call response = self.generate([inputs], run_manager=run_manager) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/michaelchu/Documents/agent/venv/lib/python3.12/site-packages/langchain/chains/llm.py", line 115, in generate return self.llm.generate_prompt( ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/michaelchu/Documents/agent/venv/lib/python3.12/site-packages/langchain_core/language_models/llms.py", line 568, in generate_prompt return self.generate(prompt_strings, stop=stop, callbacks=callbacks, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/michaelchu/Documents/agent/venv/lib/python3.12/site-packages/langchain_core/language_models/llms.py", line 741, in generate output = self._generate_helper( ^^^^^^^^^^^^^^^^^^^^^^ File "/Users/michaelchu/Documents/agent/venv/lib/python3.12/site-packages/langchain_core/language_models/llms.py", line 605, in _generate_helper raise e File "/Users/michaelchu/Documents/agent/venv/lib/python3.12/site-packages/langchain_core/language_models/llms.py", line 592, in _generate_helper self._generate( File "/Users/michaelchu/Documents/agent/venv/lib/python3.12/site-packages/langchain_core/language_models/llms.py", line 1177, in _generate self._call(prompt, stop=stop, run_manager=run_manager, kwargs) File "/Users/michaelchu/Documents/agent/venv/lib/python3.12/site-packages/langchain_community/llms/huggingface_endpoint.py", line 256, in _call response = self.client.post( ^^^^^^^^^^^^^^^^^ File "/Users/michaelchu/Documents/agent/venv/lib/python3.12/site-packages/huggingface_hub/inference/_client.py", line 242, in post hf_raise_for_status(response) File "/Users/michaelchu/Documents/agent/venv/lib/python3.12/site-packages/huggingface_hub/utils/_errors.py", line 358, in hf_raise_for_status raise BadRequestError(message, response=response) from e huggingface_hub.utils._errors.BadRequestError: (Request ID: AxsbrX3A4JxXuBdYC7fv-)

Bad request: The following model_kwargs are not used by the model: ['return_full_text', 'stop', 'watermark', 'stop_sequences'] (note: typos in the generate arguments will also show up in this list)

Description

Hi, folks. I'm just trying to run a simple LLMChain and getting the Bad Request due to model_kwargs checking. I found there are several same issue being raised, however it haven't fixed in the latest release of langchain. Please help to take a look, thanks! Previous Issue being raised: https://github.com/langchain-ai/langchain/issues/10848

System Info

System Information

OS: Darwin OS Version: Darwin Kernel Version 23.2.0: Wed Nov 15 21:53:18 PST 2023; root:xnu-10002.61.3~2/RELEASE_ARM64_T6000 Python Version: 3.12.1 (main, Feb 14 2024, 09:50:51) [Clang 15.0.0 (clang-1500.1.0.2.5)]

Package Information

langchain_core: 0.1.27 langchain: 0.1.9 langchain_community: 0.0.24 langsmith: 0.1.10

Packages not installed (Not Necessarily a Problem)

The following packages were not found:

langgraph langserve

nicole-wright commented 6 months ago

I'm facing the same problem (with various other models as well). I believe it could be caused by this commit on the huggingface_endpoint.py file. There's a new definition for _default_params that expects a base set of params for text-generation tasks. I'm guessing that if the model doesn't include those params - besides it throwing an error for models of different task types - it fails with the error you received. I could be wrong, but that's where I'm at with my research on this issue.

joelorellana commented 6 months ago

` import os from langchain_community.llms.huggingface_endpoint import HuggingFaceEndpoint

os.environ["HUGGINGFACEHUB_API_TOKEN"] = "" huggingface_llm = HuggingFaceEndpoint( repo_id="google/flan-t5-small", temperature=0, max_new_tokens=250, ) huggingface_llm("Hello!") `

Bad request: The followingmodel_kwargsare not used by the model: ['return_full_text', 'watermark', 'stop', 'stop_sequences'] (note: typos in the generate arguments will also show up in this list)

any help?

nicole-wright commented 6 months ago

I downgraded my requirements on the langchain library for now, and I can use the endpoint class. It's just a workaround, but for reference, I'm using Python 3.11, and I've pinned the following package versions:

langchain==0.1.6
langchain-cli==0.0.21
langchain-openai==0.0.6
huggingface_hub==0.21.4
python-dotenv==1.0.0
pydantic==1.10.13

hf_model_id = "facebook/bart-large-cnn"
hf_endpoint_url = f"https://api-inference.huggingface.co/models/{hf_model_id}"

llm = HuggingFaceEndpoint(
    task="summarization",
    endpoint_url=hf_endpoint_url,
)

jamesnixon-aws commented 6 months ago

I am also facing this issue, and it appears that @nicole-wright is correct. It is specifically these lines: https://github.com/langchain-ai/langchain/blob/master/libs/community/langchain_community/llms/huggingface_endpoint.py#L199-L215 which inject default params into the call to the HuggingFace API regardless of whether they are present on the HuggingFaceEndpoint instance or not.

If a model/endpoint does not support these, it causes HuggingFace to throw back an error (even if the params are populated as None, e.g. {"stop_sequences": None}).

Perhaps a simple fix would be to remove None values from the invocation_params prior to posting to the client.

Laatcha commented 6 months ago

This is happening also when using local models. I'm trying the chain with local HuggingFace model "t5-flan"

I got errors of type: The followingmodel_kwargsare not used by the model: ['return_full_text'] (note: typos in the generate arguments will also show up in this list)

llm = HuggingFacePipeline.from_model_id(model_id=flan, task="text2text-generation", model_kwargs={"temperature":1e-10
                                                                                                #    ,"return_full_text":False
                                                                                                   }, device=0)
template = PromptTemplate(input_variables=["input"], template="{input}")
chain = LLMChain(llm=llm, verbose=True, prompt=template)
chain("Say something?")

However, this is working when using local Huggingface gpt2 with "text-generation"

dongkyunlim77 commented 5 months ago

Did anyone solve this issue?

nicole-wright commented 5 months ago

I'm fairly sure it's a bug and it needs a PR. I'm not sure I know the library enough to contribute but I could always give it a shot. For the prototyping I'm doing, downgrading to a previous release works and I can still access both the hub and the pipeline objects correctly.

AccentureGabriv93 commented 5 months ago

I agree I also see the problem with various model as microsoft/phi-1_5 I hope for a fast bug fixing

saad-shahrour commented 5 months ago

I downgraded my requirements on the langchain library for now, and I can use the endpoint class. It's just a workaround, but for reference, I'm using Python 3.11, and I've pinned the following package versions:
langchain==0.1.6
langchain-cli==0.0.21
langchain-openai==0.0.6
huggingface_hub==0.21.4
python-dotenv==1.0.0
pydantic==1.10.13
hf_model_id = "facebook/bart-large-cnn"
hf_endpoint_url = f"https://api-inference.huggingface.co/models/{hf_model_id}"

llm = HuggingFaceEndpoint(
    task="summarization",
    endpoint_url=hf_endpoint_url,
)

I am trying to download the langchain-text-splitters library, but it is not suitable with these set of libraries, so I needed to upgrade the langchain library (but the main error kwargs shows again). Is there any help?? this is urgent please.

NotShrirang commented 5 months ago

I have environment like

huggingface-hub==0.22.2
langchain==0.1.15
langchain-community==0.0.32
langchain-core==0.1.42
langchain-google-genai==1.0.2
langchain-text-splitters==0.0.1
langsmith==0.1.45

I have code as below:

from langchain_community.llms.huggingface_endpoint import HuggingFaceEndpoint

pipeline = HuggingFaceEndpoint(
        huggingfacehub_api_token=os.getenv("HUGGINGFACE_API_KEY"),
        repo_id="facebook/bart-large-cnn"
)

result = pipe.invoke(doc.page_content)

And I am facing error as below:

2024-04-12 16:04:22.422 Uncaught app exception
Traceback (most recent call last):
  File "D:\Files\GitHub\RAG-for-pdf-search\venv\Lib\site-packages\huggingface_hub\utils\_errors.py", line 304, in hf_raise_for_status
    response.raise_for_status()
  File "D:\Files\GitHub\RAG-for-pdf-search\venv\Lib\site-packages\requests\models.py", line 1021, in raise_for_status        
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://api-inference.huggingface.co/models/facebook/bart-large-cnn

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "D:\Files\GitHub\RAG-for-pdf-search\app.py", line 20, in <module>
    result = pipe.invoke(doc.page_content)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\Files\GitHub\RAG-for-pdf-search\venv\Lib\site-packages\langchain_core\language_models\llms.py", line 276, in invoke
    self.generate_prompt(
  File "D:\Files\GitHub\RAG-for-pdf-search\venv\Lib\site-packages\langchain_core\language_models\llms.py", line 597, in generate_prompt
    return self.generate(prompt_strings, stop=stop, callbacks=callbacks, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\Files\GitHub\RAG-for-pdf-search\venv\Lib\site-packages\langchain_core\language_models\llms.py", line 767, in generate
    output = self._generate_helper(
             ^^^^^^^^^^^^^^^^^^^^^^
  File "D:\Files\GitHub\RAG-for-pdf-search\venv\Lib\site-packages\langchain_core\language_models\llms.py", line 634, in _generate_helper
    raise e
  File "D:\Files\GitHub\RAG-for-pdf-search\venv\Lib\site-packages\langchain_core\language_models\llms.py", line 621, in _generate_helper
    self._generate(
  File "D:\Files\GitHub\RAG-for-pdf-search\venv\Lib\site-packages\langchain_core\language_models\llms.py", line 1231, in _generate
    self._call(prompt, stop=stop, run_manager=run_manager, **kwargs)
  File "D:\Files\GitHub\RAG-for-pdf-search\venv\Lib\site-packages\langchain_community\llms\huggingface_endpoint.py", line 256, in _call
    response = self.client.post(
               ^^^^^^^^^^^^^^^^^
  File "D:\Files\GitHub\RAG-for-pdf-search\venv\Lib\site-packages\huggingface_hub\inference\_client.py", line 267, in post   
    hf_raise_for_status(response)
  File "D:\Files\GitHub\RAG-for-pdf-search\venv\Lib\site-packages\huggingface_hub\utils\_errors.py", line 358, in hf_raise_for_status
    raise BadRequestError(message, response=response) from e
huggingface_hub.utils._errors.BadRequestError:  (Request ID: 2mvdN9-F1BShz5K5FeYVk)

Bad request:
The following `model_kwargs` are not used by the model: ['stop_sequences', 'stop', 'watermark', 'return_full_text'] (note: typos in the generate arguments will also show up in this list)

I tried downgrading the langchain library, but it is causing issues with other packages (langchain-community etc.). What can I do?

anujsahani01 commented 3 months ago

Solution: Using HuggingFaceHub with LangChain

I found an effective way to use the HuggingFaceHub model with LangChain. Instead of the previous method, we can simplify and enhance the configuration as follows:

Previous Method

from langchain_huggingface.llms import HuggingFaceEndpoint 

llm = HuggingFaceEndpoint(
    repo_id="google/flan-t5-large",
    temperature=0,
    max_new_tokens=250,
    huggingfacehub_api_token=HUGGINGFACE_API_TOKEN
)

Improved Method

This new method utilizes the HuggingFaceHub from LangChain with more detailed model configurations.

from langchain import HuggingFaceHub

llm = HuggingFaceHub(
    repo_id='google/flan-t5-base',
    model_kwargs={"temperature":0, "max_length":180, 'max_new_tokens' : 120, 'top_k' : 10, 'top_p': 0.95, 'repetition_penalty':1.03}
)

# using the model 
output = llm.invoke('Can you tell me the capital of russia')

print(output)

olk commented 3 months ago

@aymeric-roucher , @baskaryan - could provide a bug fix please, your commit 0d294760e742e0707a71afc7aad22e4d00b54ae5 breaks LangChain (see bug report above)!

dordonezc commented 3 months ago

Hey, I'm also getting the same issue as mentioned above. I tried downgrading the packages with no success is there any other workaround?

jodyhuntatx commented 2 months ago

Same problem here, reverted back to HuggingFaceHub and enduring the deprecation warnings. Hopefully the bug gets fixed before the workaround is unsupported.

langchain-ai / langchain