search

huggingface / text-generation-inference

Large Language Model Text Generation Inference
http://hf.co/docs/text-generation-inference
Apache License 2.0
8.48k stars 968 forks source link

Error "Template error: template not found" #1688

Closed Ahuvim closed 2 months ago

Ahuvim commented 3 months ago

System Info

python 3.10 Langchain 0.1.4 mistral tgi hosted on ec2

Information

Tasks

Reproduction

mistral = ChatMistralAI(endpoint="http://99.80.102.238:8080/v1", mistral_api_key="mistral_api_key")
chain = Chain(database=PineconeDB())
PROMPT = PromptTemplate(template=CHAIN_TEMPLATE,
                        input_variables=["summaries", "question", "final_answer", "sources"])
chain = RetrievalQAWithSourcesChain.from_chain_type(
    llm=mistral,
    chain_type="stuff",
    retriever=chain.database.db.as_retriever(),
    chain_type_kwargs={"prompt": PROMPT},
    return_source_documents=True,
    verbose=True
)
question = ("According to media reports, at the beginning of 2024,"
            " what is the most recent information on vaccines?")

response = chain({"question": question})
print(response['answer'])

Expected behavior

When executing a query to the LLM model via LangChain with the Mistarl model hosted on EC2, I encounter a status code 422 error, accompanied by the message: {"error":"Template error: template not found","error_type":"template_error"}. Although I've reviewed the open issues related to this problem, I haven't found a clear solution applicable to my case. Any assistance in resolving this issue would be greatly appreciated.

drbh commented 3 months ago

Hi @Ahuvim would you be able to try with the latest version 2.0.1? It was just released last week 🤗

docker pull ghcr.io/huggingface/text-generation-inference:2.0.1

Additional would you be able to share what model you're using? Thank you

drbh commented 2 months ago

closing for now but please feel free to reopen is the latest version does not resolve your issue

ganisback commented 2 months ago

No work for me when I swicth to 2.0.1