ashishpatel26
commented
5 months ago It appears you're encountering an AttributeError related to the shape attribute in your code when using LangChain for question answering.
Issue
The error message indicates that an object of type StringPromptValue is being treated as if it has a shape attribute, which it does not. Specifically, the error occurs in the following lines:
batch_size, seq_length = input_ids.shapeThis suggests that input_ids should be a tensor (or an array-like object) that has a shape attribute. Instead, it is a StringPromptValue.
Likely Cause
The error likely originates from your custom RAG (Retrieval-Augmented Generation) chain, where the data being passed to the model isn't in the expected format.
Solution
- Check Data Types in the Pipeline: Ensure that all parts of the pipeline are correctly transforming the inputs and outputs, particularly before feeding data to the model.
- Modify the Custom RAG Prompt: Ensure proper transformation of the
input_idsto a format compatible with your model. Add an extra step to transform theStringPromptValueinto an appropriate tensor or array.
Implementation
Here's a potential fix:
from langchain_core.prompts import PromptTemplate
from langchain_core.runnables.base import RunnablePassthrough
from langchain_core.runnables.transform import StrOutputParser
from transformers.models.mistral.modeling_mistral import MistralModel
import torch
template = """
Use the following pieces of context to answer the question at the end.
If you don't know the answer, just say that you don't know, don't try to make up an answer.
Use three sentences maximum and keep the answer as concise as possible.
Always say "感谢提问!" at the end of the answer，总是用中文回答问题，可以使用英语描述专业词汇.
{context}
Question: {question}
Helpful Answer:"""
custom_rag_prompt = PromptTemplate.from_template(template)
rag_chain = (
{"context": retriever | format_docs, "question": RunnablePassthrough()}
| custom_rag_prompt
| cache_mistral_model
| StrOutputParser()
)
def transform_to_tensor(input_text):
# Dummy tokenizer and tensor conversion, use actual tokenizer accordingly
tokenizer = MistralModel.from_pretrained("eleutherai/mistral")
tokens = tokenizer.encode(input_text, return_tensors="pt")
return tokens
while True:
user_input = input("请输入问题或命令（输入 q 退出）: ")
if user_input.lower() == "q":
break
# Transform user input to correct type for model processing
transformed_input = transform_to_tensor(user_input)
for chunk in rag_chain.stream(transformed_input):
print(chunk, end="", flush=True)Explanation
transform_to_tensorFunction: Transformsuser_inputinto a tensor that the model can work with.- Tokenizer: Demonstrates dummy tokenizer usage. Replace it with your model's actual tokenizer.
Notes
- Ensure all dependencies are correctly installed.
- Adapt the transformation function to your specific model/tokenizer if needed.
- This is a basic example to get you started; you might need further adjustments based on your specific pipeline configuration.
Additional Resources
- LangChain Documentation: Refer to the official documentation for detailed guidance.
Analysis
Carefully check and adjust the data types and transformations in the pipeline to resolve the AttributeError. This example provides a framework for addressing the issue by transforming inputs to the correct format expected by the model.
ZephryLiang
URL
https://python.langchain.com/v0.2/docs/tutorials/rag/
Checklist
Issue with current documentation:
mycode `
error say :Traceback (most recent call last): File "/home/desir/PycharmProjects/pdf_parse/rag/cohere.py", line 141, inwhat happened?please
Idea or request for content:
i don't know