Translate User Query and Model Response in RetrievalQA Chain

[weissenbacherpwc]

Hi,

I have built a RAG app with RetrievalQA and now wanted to try out a new approach. I am using an English LLM but the responses should be in German. E.g. if the user asks something in German "Hallo, wer bist du?", the user query should be translated to "Hello, who are you?" before feeding it into the RAG pipeline. After the model made its response in English "I am an helpful assistant" the output should be translated back to German "Ich bin ein hilfreicher Assistent".

As translator I am using googletrans==3.1.0a0

Here is my RetrievalQA Chain:

from langchain.chains import RetrievalQA
from langchain.memory import ConversationBufferWindowMemory
import box
import yaml
from src.utils import set_prompt, setup_dbqa, build_retrieval_qa
from src.llm import build_llm
from src.prompts import mistral_prompt
from langchain.vectorstores import FAISS

with open('config/config.yml', 'r', encoding='utf8') as ymlfile:
    cfg = box.Box(yaml.safe_load(ymlfile))

def build_retrieval_qaa(llm, prompt, vectordb):
    chain_type_kwargs={ "prompt": prompt,
                        "memory": ConversationBufferWindowMemory(
                            memory_key="chat_history",
                            input_key="question",
                            #output_key="answer",
                            k=8,
                            return_messages=True),
                        "verbose": False
                        }
    dbqa = RetrievalQA.from_chain_type(llm=llm,
                                        chain_type='stuff',
                                        retriever=vectordb,
                                        return_source_documents=cfg.RETURN_SOURCE_DOCUMENTS,
                                        chain_type_kwargs=chain_type_kwargs,
                                        verbose=False
                                        )
    return dbqa

llm = build_llm(ANY LLM)
qa_prompt = set_prompt(mistral_prompt)
vectordb = FAISS.load_local(cfg.DB_FAISS_PATH, bge_embeddings)
vectordb = vectordb.as_retriever(search_kwargs={'k': cfg.VECTOR_COUNT, 'score_treshold': cfg.SCORE_TRESHOLD}, search_type="similarity")
dbqa = build_retrieval_qaa(llm, qa_prompt, vectordb)
dbqa("Was bedeutet IPv6 für die Software-Entwicklung?") # Gives me a response

The prompt looks like this:

mistral_prompt = """
<s> [INST] Du bist RagBot, ein hilfsbereiter Assistent. Antworte nur auf Deutsch. Verwende die folgenden Kontextinformationen, um die Frage am Ende zu beantworten. Wenn du die Antwort nicht kennst, sag einfach, dass du es nicht weisst. Versuche nicht eine Antwort zu erfinden.

###Chat History###: {chat_history}

###Kontext###: {context} 

###Frage###: {question} 
Antwort: [/INST]
"""

So what do I have to change here, to first translate the user query and the prompt from DE to EN, and afterwards the Model response from EN to DE? Specifically I have problems to translate the provided context, chat history and question.

[dosubot[bot]]

🤖

Hallo @weissenbacherpwc! Ich bin Dosu, ein Bot, der Ihnen hilft, Fehler zu beheben, Fragen zu beantworten und ein Mitwirkender zu werden. Während Sie auf einen menschlichen Maintainer warten, bin ich hier, um Ihnen zu helfen. Lassen Sie uns gemeinsam diese Herausforderung bewältigen!

To modify your RetrievalQA Chain in LangChain to translate user queries and model responses between German and English using googletrans==3.1.0a0, you can override the _call and _acall methods in the RetrievalQA class.

First, you need to import the Translator class from googletrans:

from googletrans import Translator

Then, create an instance of the Translator class:

translator = Translator()

In the _call method, translate the user's query from German to English before calling _get_docs and translate the model's response from English to German before returning it:

def _call(
    self,
    inputs: Dict[str, Any],
    run_manager: Optional[CallbackManagerForChainRun] = None,
) -> Dict[str, Any]:
    _run_manager = run_manager or CallbackManagerForChainRun.get_noop_manager()
    question = inputs[self.input_key]
    question = translator.translate(question, src='de', dest='en').text
    accepts_run_manager = (
        "run_manager" in inspect.signature(self._get_docs).parameters
    )
    if accepts_run_manager:
        docs = self._get_docs(question, run_manager=_run_manager)
    else:
        docs = self._get_docs(question)  # type: ignore[call-arg]
    answer = self.combine_documents_chain.run(
        input_documents=docs, question=question, callbacks=_run_manager.get_child()
    )
    answer = translator.translate(answer, src='en', dest='de').text

    if self.return_source_documents:
        return {self.output_key: answer, "source_documents": docs}
    else:
        return {self.output_key: answer}

Do the same for the _acall method:

async def _acall(
    self,
    inputs: Dict[str, Any],
    run_manager: Optional[AsyncCallbackManagerForChainRun] = None,
) -> Dict[str, Any]:
    _run_manager = run_manager or AsyncCallbackManagerForChainRun.get_noop_manager()
    question = inputs[self.input_key]
    question = translator.translate(question, src='de', dest='en').text
    accepts_run_manager = (
        "run_manager" in inspect.signature(self._aget_docs).parameters
    )
    if accepts_run_manager:
        docs = await self._aget_docs(question, run_manager=_run_manager)
    else:
        docs = await self._aget_docs(question)  # type: ignore[call-arg]
    answer = await self.combine_documents_chain.arun(
        input_documents=docs, question=question, callbacks=_run_manager.get_child()
    )
    answer = translator.translate(answer, src='en', dest='de').text

    if self.return_source_documents:
        return {self.output_key: answer, "source_documents": docs}
    else:
        return {self.output_key: answer}

Please note that the translation process might slow down the response time of your application. You can find more details about the RetrievalQA class in the LangChain repository.

Sources

• libs/langchain/langchain/chains/retrieval_qa/base.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.