generate() got multiple keyword arguments for 'stop_sequences' while generating using hf TGI

[adarshxs]

System Info

I'm not sure what exactly is causing the issue. Is it langchain, TGI, or streamlit

generate() got multiple keyword arguments for 'stop_sequences' upon running the script along with streamlit

TypeError: generate() got multiple values for keyword argument 'stop_sequences'
Traceback:
File "/usr/local/lib/python3.8/dist-packages/streamlit/runtime/scriptrunner/script_runner.py", line 552, in _run_script exec(code, module.dict)
File "/workspace/app.py", line 154, in main()
File "/workspace/app.py", line 109, in main handle_userinput(user_question)
File "/workspace/app.py", line 79, in handle_userinput response = st.session_state.conversation({'question': user_question})
File "/usr/local/lib/python3.8/dist-packages/langchain/chains/base.py", line 243, in call raise e
File "/usr/local/lib/python3.8/dist-packages/langchain/chains/base.py", line 237, in call self._call(inputs, run_manager=run_manager)
File "/usr/local/lib/python3.8/dist-packages/langchain/chains/conversational_retrieval/base.py", line 142, in _call answer = self.combine_docs_chain.run(
File "/usr/local/lib/python3.8/dist-packages/langchain/chains/base.py", line 445, in run return self(kwargs, callbacks=callbacks, tags=tags, metadata=metadata)[
File "/usr/local/lib/python3.8/dist-packages/langchain/chains/base.py", line 243, in call raise e
File "/usr/local/lib/python3.8/dist-packages/langchain/chains/base.py", line 237, in call self._call(inputs, run_manager=run_manager)
File "/usr/local/lib/python3.8/dist-packages/langchain/chains/combine_documents/base.py", line 106, in _call output, extra_return_dict = self.combine_docs(
File "/usr/local/lib/python3.8/dist-packages/langchain/chains/combine_documents/stuff.py", line 165, in combine_docs return self.llm_chain.predict(callbacks=callbacks, **inputs), {}
File "/usr/local/lib/python3.8/dist-packages/langchain/chains/llm.py", line 252, in predict return self(kwargs, callbacks=callbacks)[self.output_key]
File "/usr/local/lib/python3.8/dist-packages/langchain/chains/base.py", line 243, in call raise e
File "/usr/local/lib/python3.8/dist-packages/langchain/chains/base.py", line 237, in call self._call(inputs, run_manager=run_manager)
File "/usr/local/lib/python3.8/dist-packages/langchain/chains/llm.py", line 92, in _call response = self.generate([inputs], run_manager=run_manager)
File "/usr/local/lib/python3.8/dist-packages/langchain/chains/llm.py", line 102, in generate return self.llm.generate_prompt(
File "/usr/local/lib/python3.8/dist-packages/langchain/llms/base.py", line 188, in generate_prompt return self.generate(prompt_strings, stop=stop, callbacks=callbacks, **kwargs)
File "/workspace/app.py", line 18, in generate super().generate(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/langchain/llms/base.py", line 281, in generate output = self._generate_helper(
File "/usr/local/lib/python3.8/dist-packages/langchain/llms/base.py", line 225, in _generate_helper raise e
File "/usr/local/lib/python3.8/dist-packages/langchain/llms/base.py", line 212, in _generate_helper self._generate(
File "/usr/local/lib/python3.8/dist-packages/langchain/llms/base.py", line 604, in _generate self._call(prompt, stop=stop, run_manager=run_manager, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/langchain/llms/huggingface_text_gen_inference.py", line 156, in _call res = self.client.generate(

from langchain.llms.huggingface_text_gen_inference import HuggingFaceTextGenInference
from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler

llm = HuggingFaceTextGenInference(
    inference_server_url='http://127.0.0.1:8080',
    max_new_tokens=512,
    top_k=10,
    top_p=0.95,
    typical_p=0.95,
    temperature=0.85,
    stop_sequences=['</s>'],
    repetition_penalty=1.03,
    stream=True
)
print(llm("What is a proctor?", callbacks=[StreamingStdOutCallbackHandler()]))

the above script works properly but while using it with chainlit I run into the generate() got multiple keyword arguments for 'stop_sequences'

Who can help?

@hwchase17 @baskaryan

Information

• [ ] The official example notebooks/scripts
• [ ] My own modified scripts

Related Components

• [X] LLMs/Chat Models
• [ ] Embedding Models
• [ ] Prompts / Prompt Templates / Prompt Selectors
• [ ] Output Parsers
• [ ] Document Loaders
• [ ] Vector Stores / Retrievers
• [ ] Memory
• [ ] Agents / Agent Executors
• [ ] Tools / Toolkits
• [X] Chains
• [ ] Callbacks/Tracing
• [ ] Async

Reproduction

from langchain.llms.huggingface_text_gen_inference import HuggingFaceTextGenInference
from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler

llm = HuggingFaceTextGenInference(
    inference_server_url='http://127.0.0.1:8080',
    max_new_tokens=512,
    top_k=10,
    top_p=0.95,
    typical_p=0.95,
    temperature=0.85,
    stop_sequences=['</s>'],
    repetition_penalty=1.03,
    stream=True
)
print(llm("What is a proctor?", callbacks=[StreamingStdOutCallbackHandler()]))

the above script works properly but while using it with chainlit I run into the generate() got multiple keyword arguments for 'stop_sequences'

Expected behavior

generate normally

[dosubot[bot]]

Answer generated by a 🤖

Answer

From your description, it appears that you're encountering an error message stating "generate() got multiple keyword arguments for 'stop_sequences'" when running your script with Streamlit. This error typically suggests that the 'stop_sequences' argument is being passed more than once to the 'generate' function.

Looking at the traceback, the error seems to originate from the 'generate' function in the 'HuggingFaceTextGenInference' class within the 'langchain/llms/huggingface_text_gen_inference.py' file. The 'stop_sequences' argument is passed to the 'generate' function via the 'invocation_params' dictionary in the '_call' method. If 'stop_sequences' is also included in the '**kwargs' when calling the 'generate' function, it would be passed twice, leading to the error you're seeing.

To resolve this issue, you should ensure that 'stop_sequences' is not included in the '**kwargs' when calling the 'generate' function. If it is, you'll need to remove it before calling 'generate'.

Please review your code to ensure 'stop_sequences' is not being passed twice to the 'generate' function. If you're using the 'TextGen' class or the 'generate_prompt' and '_generate_with_cache' methods in your code, make sure that 'stop_sequences' is not being passed as an argument to both 'HuggingFaceTextGenInference' and these classes or methods, as this could also cause the error.

This is one potential solution based on the information provided. If this doesn't resolve your issue, or if your experience contradicts this solution, it might be helpful to ensure you're using the latest version of the LangChain framework.

If you believe this is a bug that could impact other users and you're comfortable doing so, you're encouraged to make a pull request with the necessary changes. Your contribution would be greatly appreciated by the LangChain community.

Sources

• langchain/llms/huggingface_text_gen_inference.py
• tests/integration_tests/llms/test_huggingface_text_gen_inference.py
• langchain/llms/textgen.py
• langchain/chat_models/base.py
• langchain/llms/rwkv.py
• langchain/llms/ctransformers.py
• langchain/chat_models/openai.py

---

This response is meant to be useful, save you time, and share context. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

[adarshxs]

Was solved in #8044