keenborder786
commented
2 months ago Please try the following:
from langchain_community.llms.huggingface_endpoint import HuggingFaceEndpoint
from langchain.pydantic_v1 import BaseModel,Field,validator
from langchain.prompts import PromptTemplate
from langchain.output_parsers import PydanticOutputParser
from typing import List
repo_id = "mistralai/Mistral-7B-Instruct-v0.3"
model_kwargs = {
"max_new_tokens": 100,
"max_length": 200,
"temperature": 0.1,
"timeout": 6000
}
# Using HuggingFaceHub
llm = HuggingFaceEndpoint(
repo_id=repo_id,
huggingfacehub_api_token = '',
task = 'conversational',
**model_kwargs,
)
# Define your desired data structure.
class Suggestions(BaseModel):
words: List[str] = Field(description="list of substitute words based on context")
# Throw error in case of receiving a numbered-list from API
@validator('words')
def not_start_with_number(cls, field):
for item in field:
if item[0].isnumeric():
raise ValueError("The word can not start with numbers!")
return field
parser = PydanticOutputParser(pydantic_object=Suggestions)
prompt_template = """
Please output your answer as a JSON following the below instruction:
{format_instructions}
Offer a list of suggestions to substitute the specified target_word based on the context.
target_word={target_word}
context={context}
"""
prompt_input_variables = ["target_word", "context"]
partial_variables = {"format_instructions":parser.get_format_instructions()}
prompt = PromptTemplate(
template=prompt_template,
input_variables=prompt_input_variables,
partial_variables=partial_variables
)
chain = prompt | llm
response = chain.invoke({'target_word':'behaviour',
'context':'The behaviour of the students in the classroom was disruptive and made it difficult for the teacher to conduct the lesson.'})
print(parser.parse(response))
- Updated the Prompt
- Use the updated
HuggingFaceEndpointrather than the deprecatedHuggingFaceHub - Change the task type
- Use the new LCEL style
daniau23
Checked other resources
Example Code
The following is the code when using PydanticOutputParser as Langchain fails to parse LLM output
When trying to fix the error using OutputFixingParser another error was experienced below is the codebase
Error Message and Stack Trace (if applicable)
Error when using OutputFixingParser
Description
Output must be able to parse LLM output and extract the json produced as shown below;
System Info
System Information
Package Information
Packages not installed (Not Necessarily a Problem)
The following packages were not found: