Problem in Answers and Retrieval document using LLAMA-2 13 B model with langchain

[tripurarisingh1]

Issue with current documentation:

I am using LLAMA-2 13 B model with langchain For embeddings i am using embeddings = HuggingFaceInstructEmbeddings( model_name="WhereIsAI/UAE-Large-V1", model_kwargs={"device": DEVICE} )

db = FAISS.load_local(path, embeddings,allow_dangerous_deserialization=True) prompt_template = f"{template}\nCONTEXT:\n\n{{context}}\nQuestion: {{question}}\n[INST]" prompt = PromptTemplate(template=prompt_template, input_variables=["context", "question"])

       qa_chain = RetrievalQA.from_chain_type(
            llm=llm2,
            chain_type="stuff",
            retriever=db.as_retriever(search_kwargs={"k": 4}),
            return_source_documents=True,
            chain_type_kwargs={"prompt": prompt}
        )

result = qa_chain({"context": "", "query": user_input+" Just tell what you know"})

Chunk Size: text_splitter = RecursiveCharacterTextSplitter(chunk_size=512, chunk_overlap=60) texts = text_splitter.split_documents(docs)

For question/answers it's not providing correct retriever document and answers.

Kindly provide me answers how i need to fix this. it's high prority.

Idea or request for content:

No response

[tripurarisingh1]

Hi All,

Any update on it ?

[keenborder786]

@tripurarisingh1 you should create your chain as follow:

  embeddings = HuggingFaceInstructEmbeddings(
  model_name="WhereIsAI/UAE-Large-V1", model_kwargs={"device": DEVICE}
  )

  db = FAISS.load_local(path, embeddings,allow_dangerous_deserialization=True)
  from langchain_community.llms import OpenAI
  from langchain.chains import RetrievalQA
  from langchain_community.vectorstores import FAISS
  from langchain_core.vectorstores import VectorStoreRetriever
  retriever = VectorStoreRetriever(vectorstore=db)
  retrievalQA = RetrievalQA.from_llm(llm=llm2, retriever=retriever)

I don't understand what you are using the Spiliting for if you are loading FAISS from local index.