Apply LLM Embedder

[QuangTQV]

Description

I am working on a blockchain chatbot and applying LLM embedder to retrieve tools and news. However, there is a bottleneck: these retrievals can only be performed with a specific query. Sometimes users ask questions in a vague manner, requiring us to review the conversation history to understand the context. For example, if a user asks, "What is its price?" it is not possible to search with such an unclear query.

Problem

I have an idea to refine the current user's query into a standalone, clear query based on the conversation history. However, it seems that LLM GPT 3.5 Turbo does not perform well in this task. Is there a better solution for this issue?

Additional Concerns

I also do not understand how the "chat" and "convsearch" tasks of the LLM embedder can be practically applied. They seem to search based on the conversation session. In a real chatbot system, users will ask first, and we will use the conversation history and the current query of user to search (At this time there is no bot answer to the current user's request). However, when I look at the sample data of "chat" and "convsearch", I only see queries and answers from conversation sessions and do not see the current query of user.

[namespace-Pt]

Hi, thanks for your interest.

• LLM-Embedder is able to encode the conversation context as the input query because it has been trained on QReCC, a conversational search dataset. You can try concatenating the conversation context as the input query. Using an LLM to rewrite a context-independent query is another approach. I think GPT 3.5 should excel on this task. Maybe you can try some different prompts.
• The current question of the user is in the query field.

[QuangTQV]

Hi, thanks for your interest.

• LLM-Embedder is able to encode the conversation context as the input query because it has been trained on QReCC, a conversational search dataset. You can try concatenating the conversation context as the input query. Using an LLM to rewrite a context-independent query is another approach. I think GPT 3.5 should excel on this task. Maybe you can try some different prompts.
• The current question of the user is in the query field.

My point is that I used LLM-Embedder to retrieve relevant dialogue history based on the current user input and the latest dialogue pairs. However, it seems like it only retrieves dialogue history related to the latest dialogue pairs while ignoring segments related to the current user input. I think this phenomenon occurs because the current user input is very short compared to the latest dialogue pairs.

[QuangTQV]

Can anyone help me?

[namespace-Pt]

Hi, maybe you need to further fine-tune the model to retrieve based on the user input.