search

microsoft / promptflow

Build high-quality LLM apps - from prototyping, testing to production deployment and monitoring.
https://microsoft.github.io/promptflow/
MIT License
9.28k stars 840 forks source link

Deploy flow to Azure managed online endpoint #356

Closed benjaminbluhm closed 11 months ago

benjaminbluhm commented 1 year ago

Is your feature request related to a problem? Please describe. Is it possible to use sdk/cli to deploy flow to Azure managed online endpoint? This would be very useful for CI/CD process

Describe the solution you'd like If possible would be great to have an example in the docs (similar to azure app service deployment example)

ttthree commented 1 year ago

@wangchao1230 to follow up as a documentation improvement.

eniac871 commented 1 year ago

Ack the ask, we have some code improvement plan to make this path work smoothly. It will take some time, we will share the code and samples when that's ready.

varuna82 commented 1 year ago

Is it possible to deploy Prompt Flow to Azure Functions? Since heavy lifting is done by the GPT endpoints, Azure functions should be enough to handle lightweight callers.

benjaminbluhm commented 11 months ago

I understand the implementation of this feature will take some time as mentioned by @eniac871 and also that it is possible to manually deploy a flow to an online endpoint.

Eagerly waiting for this feature to be implemented and documented in promptflow since I need to automate the deployment in a pipeline.

In the meantime, can anyone indicate if the below path should work?

  1. Build the docker image as referenced in the promptflow deployment guide
  2. Deploy the flow by following along the Azure ML guidelines for using a custom container to deploy a model to an online endpoint

Would be glad about any high-level guidance of how to automate the deployment workflow - if this is not possible or not recommended for some reason at the moment, this would also be a very valuable information.

Thank you!

benjaminbluhm commented 11 months ago

the process is now documented here:

https://learn.microsoft.com/en-us/azure/machine-learning/prompt-flow/how-to-end-to-end-llmops-with-prompt-flow?view=azureml-api-2