Unclear how /extensions/chat/completions does the retrieval behind the scenes

[gingergenius]

I have failed to understand from the documentation how the "/deployments/{deployment-id}/extensions/chat/completions" endpoint interacts with Cognitive Search behind the scenes. The background is I'm trying to understand what flexibility it offers and what it would take to implement the retrieval and integration of documents into the LLM's prompt manually if we want to change something.

What Cognitive Search endpoint does the extension call and with what parameters? Here's an example of an API request I sent myself to try reproducing the top 5 results within the tool citations

curl --location 'https://[deployment].search.windows.net/indexes/[index]/docs/search?api-version=2023-07-01-Preview' \
--header 'Content-Type: application/json' \
--header 'api-key: [key]' \
--data '{  
     "queryType": [I tried full, simple and semantic here, semantic with different settings for other required parameters]
     "search": "[question text]",  
     "top": 5
   }  '

I am getting the same documents back as in the Search Explorer for Cognitive Search in the Azure portal, but they are different from what comes back from the extensions/chat/completions request. The relevance scores are sometimes the same for the same chunks, but sometimes also different. Could you unveil why that happens?

Is it correct that no embeddings are used in the document retrieval as implemented in the Azure OpenAI playground and this sample app?

Is there more system text hidden away somewhere instructing the model to look at the sources and provide references in this [doc1] format? How would we go about modifying that if we are not happy with the citation accuracy?

---

Document Details

⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

• ID: fadd6778-b11c-d9af-1bd9-e3ed394e0fcb
• Version Independent ID: 45436c53-4aae-284b-a6c5-3f47c9342537
• Content: Azure OpenAI Service REST API reference - Azure OpenAI
• Content Source: articles/ai-services/openai/reference.md
• Service: cognitive-services
• Sub-service: openai
• GitHub Login: @mrbullwinkle
• Microsoft Alias: mbullwin

[AjayBathini-MSFT]

@gingergenius Thanks for your feedback! We will investigate and update as appropriate.

[RamanathanChinnappan-MSFT]

@gingergenius

I've delegated this to @mrbullwinkle, a content author, to review and share their valuable insights.

[mrbullwinkle]

We have moved away from "deployments/{deployment-id}/extensions/chat/completions" in the latest stable/preview API releases so this is not a topic that we will be able to cover in depth in the docs in the future.

[mrbullwinkle]

please-close