Azure-Samples / azure-search-openai-demo

A sample app for the Retrieval-Augmented Generation pattern running in Azure, using Azure AI Search for retrieval and Azure OpenAI large language models to power ChatGPT-style and Q&A experiences.
https://azure.microsoft.com/products/search
MIT License
5.69k stars 3.83k forks source link

Connect user with their Onedrive/Sharepoint data #1563

Open mo-albaba opened 3 months ago

mo-albaba commented 3 months ago

Please provide us with the following information: I have AZURE_USE_AUTHENTICATION true and want the user to be able to search their Onedrive/SharePoint and Mailbox data. How can I connect live API with these resources and be searchable after the user authenticates?

This issue is for a: (mark with an x)

- [ ] bug report -> please search issues before submitting
- [ x] feature request
- [ ] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

azd version?

azd version 1.8.1

Mention any other details that might be useful

The users on my environment are hosted on Microsoft Entra ID

pamelafox commented 3 months ago

This repository does not currently have any particular integration with Sharepoint. You may want to read through related issues to see if developers have shared their approaches: https://github.com/Azure-Samples/azure-search-openai-demo/issues?q=is%3Aissue+is%3Aopen+sharepoint

You could consider using the Sharepoint API: https://learn.microsoft.com/en-us/sharepoint/dev/sp-add-ins/get-to-know-the-sharepoint-rest-service?tabs=csom

If you're using the authentication option, then presumably you could ask for the scope that'd give access to the user's relevant data, and then you could do a search on the user's behalf, and send them to the LLM.

Or you could use the Sharepoint indexer for getting data into Azure AI search, if you're doing RAG on widely available documents, but it sounds like that's not the case. https://learn.microsoft.com/en-us/azure/search/search-howto-index-sharepoint-online

mo-albaba commented 2 months ago

The first link isn't taking me to a specific issue. Also if I want to add more capability for the chat app search index like the SharePoint API, how and where will this added scripting go on the main workspace?

pamelafox commented 2 months ago

I linked to a search for "sharepoint" in the issue tracker since it has come up in multiple threads.

To additionally query the Sharepoint API, you'd want to modify the files in the approaches folders. For example, for the chat tab, you'd modify this file: https://github.com/Azure-Samples/azure-search-openai-demo/blob/96ec028281da79f40b3b87e776e7447f70d79660/app/backend/approaches/chatreadretrieveread.py#L157

I'm not sure if you're imagining replacing the search entirely, or optionally doing search based on what the user searched for. You may want to read more about function calling, as you might want to add a function or add a parameter to the existing function that's used during the query rewriting phrase. See this blog post: https://blog.pamelafox.org/2024/03/rag-techniques-using-function-calling.html