Answer Engine Quality - Ideas

[wsxiaoys]

Under Construction

The Answer Engine, released in version 0.13, provides a Q&A interface for Tabby's users to interact with the LLM, optionally within the context of a connected repository. The implementation is quite naive thus not really an performing one, we'd like to record ideas and improve the quality upon current baseline.

Prompt construction

Currently, the implementation is relatively simple. We collect snippets(instead of full article) from various sources and input them into a single LLM inference call to generate both the answer and the grounding (citation) information simultaneously.

1. collect references: Query the index, which includes both code and documents, to gather chunks. Deduplication is carried out at the document level, making sure that from each document, only the chunk with the highest score is chosen.

2. answer generation: Construct the prompt by combining the question and the content of each chunk to create a single LLM inference prompt.

3. relevant questions: Generate relevant questions for the request, the prompt is also based on question and chunk

Ideas

• [ ] #3096
• [ ] #3122
• [ ] #3123

Integrations

• [ ] #2894
• [ ] #3097
• [x] #2984

[shinohara-rin]

I always though it's a good idea to not rely solely on embedding models for the retrival of relevant code.

In reality, human developers relies a lot on language servers, it would make sense to let LLMs make use of the language servers too (autocompletion, retriving def/refs, type inference, arguments hinting, docs, etc.)

For instance, extracting all symbols inside the user's selection and feeding the definitions and/or references to the LLM. This is of course a rather naive approach, but it should facilitate code understanding of the LLM.