ConversationalRetrievalChain did not look into chat histroy while making an answer

I am trying to make a chatbot which remember the existing chat and can answer from that as well as from document. Here is what I have tried.

memory = ConversationBufferMemory(memory_key="chat_history", return_messages=True)
qa = ConversationalRetrievalChain.from_llm(
                                        ChatOpenAI(model_name='gpt-3.5-turbo'),
                                        retriever=docsearch, 
                                        memory=memory,
                                        verbose=True)

result = qa({"question": "My name is Talha. Ali is my friend. What is CNN?"})

My document is based on CNN, so it fetched what is CNN Then I ask another question.

result = qa({"question": "Who is Ali?"})

this is what happened behind the scene

> Entering new LLMChain chain...
Prompt after formatting:
Given the following conversation and a follow up question, rephrase the follow up question to be a standalone question, in its original language.

Chat History:

Human: My name is Talha. Ali is my friend. What is CNN?
Assistant: CNN is a tool for deep learning and machine learning algorithms used in artificial neural networks for image recognition, object detection, and segmentation.
Follow Up Input: Who is Ali?
Standalone question:

> Finished chain.

> Entering new StuffDocumentsChain chain...

> Entering new LLMChain chain...
Prompt after formatting:
Use the following pieces of context to answer the question at the end. If you don't know the answer, just say that you don't know, don't try to make up an answer.

_few paragraphs from documents i have ingested in vector database_

Question: Ali is Talha's friend.
Answer:

But when i print result['answer'] It say "I don't know."

Here is notebook for reproducibility https://colab.research.google.com/drive/1UTjdpAAjoZ_ccpdAEr9gpfpqlC3pwtmT?usp=sharing

langchain-ai / langchain

ConversationalRetrievalChain did not look into chat histroy while making an answer #5984