Azure-Samples / chat-with-your-data-solution-accelerator

A Solution Accelerator for the RAG pattern running in Azure, using Azure AI Search for retrieval and Azure OpenAI large language models to power ChatGPT-style and Q&A experiences. This includes most common requirements and best practices.

https://azure.microsoft.com/products/search

MIT License

651 stars 324 forks source link

Add the PromptFlow from "On Your Data" and check it works with our index #915

Closed gaurarpit closed 3 weeks ago

gaurarpit commented 1 month ago

Required by https://github.com/Azure-Samples/chat-with-your-data-solution-accelerator/issues/406

Description

Create and deploy a flow which mimics the current converstation flow.

Tasks

[x] Spike replicating the behavior in prompt flow and how to deploy
[x] Create flow
[x] Deploy the flow via Bicep - https://github.com/Azure-Samples/chat-with-your-data-solution-accelerator/pull/1003

adamdougal commented 1 month ago

Update 20th May:

Currently working on spiking the implementation
We've successfully replicated orchestration flow in prompt flow
We've successfully exported the flow and run this locally using pf flow serve ...
We're hitting an issue when trying to build the flow using pf flow build...
- We need this to be able to deploy it to Azure App Service
- The issue is that the Azure AI Search connection is not being picked up by the pf flow build command
- We've raised an issue here as I believe it is a bug with the pf tool https://github.com/microsoft/promptflow/issues/3313
- We'll keep digging for solutions

adamdougal commented 1 month ago

Further Update 20th May:

I've managed to get around the issue with the AI Search connection not being picked up by adding connection: aisearch_connection to the lookup node
I'm now running into another issue where running the built executable works, but sending queries through it results in a 401 -Unauthorized. Access token is missing, invalid, audience is incorrect (https://cognitiveservices.azure.com), or have expired. error when calling OpenAI.
- I've checked the Key is correct and all is good
- I'll continue to investigate

adamdougal commented 1 month ago

Update 21st May:

After receiving guidance from the prompt flow team, I have been able to deploy the flow to an AML endpoint
Discussed this with the team and this is more desirable than deploying to an App Service as it's closer to where the prompt flow is defined and experimentation is run
Next step is to work out how to deploy an AML endpoint using bicep

adamdougal commented 1 month ago

Update 24th May:

I've got the workspace and connections deploying successfully using Bicep
However, it does not seem to be possible to deploy a flow and a corresponding endpoint using Bicep
I've explored using deployment scripts, but these run on a container in Azure, it's not trivial to install the required dependencies (pfazure) and it does not have access to local files.
As a fallback, we could look at using a azd postprovision hook, which runs locally to deploy the flow. The main downside of this, is that there would be a manual step for users who deploy using one click deploy.
I'll discuss with the team to decide next steps.