Pipeline with tokenizer without pad_token cannot do batching. You can try to set it with `pipe.tokenizer.pad_token_id = model.config.eos_token_id`.

Checked other resources

[X] I added a very descriptive title to this issue.
[X] I searched the LangChain documentation with the integrated search.
[X] I used the GitHub search to find a similar question and didn't find it.
[X] I am sure that this is a bug in LangChain rather than my code.
[X] The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

from langchain_huggingface import ChatHuggingFace, HuggingFacePipeline from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = 'meta-llama/Meta-Llama-3.1-8B-Instruct'

llm = HuggingFacePipeline.from_model_id( model_id=model_id, task="text-generation", device=None, model_kwargs=dict( device_map="auto", ), pipeline_kwargs=dict( temperature=0.6, max_new_tokens=512, repetition_penalty=1.1 ), )

chat_model = ChatHuggingFace(llm=llm)

from langchain_core.messages import ( HumanMessage, SystemMessage, )

messages = [ SystemMessage(content="You're a helpful assistant"), HumanMessage( content="What happens when an unstoppable force meets an immovable object?" ), ]

ai_msg = chat_model.invoke(messages)

Error Message and Stack Trace (if applicable)

ValueError Traceback (most recent call last) Cell In[20], line 35 23 from langchain_core.messages import ( 24 HumanMessage, 25 SystemMessage, 26 ) 28 messages = [ 29 SystemMessage(content="You're a helpful assistant"), 30 HumanMessage( 31 content="What happens when an unstoppable force meets an immovable object?" 32 ), 33 ] ---> 35 ai_msg = chat_model.invoke(messages)

File ~/miniconda3/lib/python3.12/site-packages/langchain_core/language_models/chat_models.py:277, in BaseChatModel.invoke(self, input, config, stop, kwargs) 266 def invoke( 267 self, 268 input: LanguageModelInput, (...) 272 kwargs: Any, 273 ) -> BaseMessage: 274 config = ensure_config(config) 275 return cast( 276 ChatGeneration, ... 152 ) 153 else: 154 t_padding_value = tokenizer.pad_token_id

ValueError: Pipeline with tokenizer without pad_token cannot do batching. You can try to set it with pipe.tokenizer.pad_token_id = model.config.eos_token_id.

Description

I am trying to use langchain chat model with meta-llama/Meta-Llama-3.1-8B-Instruct, but cannot init a chat model, with the tokenizer issue.

System Info

langchain==0.2.16 langchain-community==0.2.16 langchain-core==0.2.38 langchain-huggingface==0.0.3 langchain-text-splitters==0.2.4

langchain-ai / langchain