microsoft / azurechat

🤖 💼 Azure Chat Solution Accelerator powered by Azure Open AI Service
MIT License
1.14k stars 954 forks source link

Search Extension Based on AI Search Returning Incomplete or Incorrect Citations #347

Open lbregmand opened 3 months ago

lbregmand commented 3 months ago

When setting up a search extension based on AI Search, some citations return as either empty or refer to non-existing files along with their respective names or IDs. Often, the issue arises when the file name has been modified by LLM, which interferes with the reference.

The citation service is utilized within src/features/chat-page/chat-services/citation-service.ts and applies the citation format defined in the extension description (as per the document docs/6-chat-over-file.md), :

'You are an expert in searching internal documents using aisearch function. You should always include a citation at the end of your answer but refrain from including a full stop after the citations.

The format for your citation should be:

{% citation items=[{name:"filename 1",id:"file id"}, {name:"filename 2",id:"file id"}] %}'

However, it is observed that stability issues surface regularly with the citation service and there are frequent data inconsistencies or missing entries. Prompt improvement can bring limited benefit, but we are looking for better solution without prompt.