search

MicrosoftDocs / azure-docs

Open source documentation of Microsoft Azure
https://docs.microsoft.com/azure
Creative Commons Attribution 4.0 International
10.2k stars 21.35k forks source link

BCDR question for online endpoint with multiple deployments #95314

Open sunnyhay opened 2 years ago

sunnyhay commented 2 years ago

Hi, is there any documentation/best practice to enable a single online endpoint with multiple deployments in different regions for failover consideration? For instance, using the same model/API a single endpoint can allow blue deployment on East US while green deployment on West US. If this does not work, I plan to build multiple endpoints against the same set of model/API and still need some routing/load balancing solution to route traffic among these endpoints. Is Azure Traffic Manager available for this purpose? Or some other options, e.g. Azure Application Gateway. Thank you!

Document Details

⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

YutongTie-MSFT commented 2 years ago

Thanks for the feedback! We are currently investigating and will update you shortly.

sunnyhay commented 2 years ago

Thanks. Any update so far?

YutongTie-MSFT commented 2 years ago

Hello @dem108 I don't find any official doc for this part, could you please help?

@sunnyhay Thanks for the feedback! I have assigned the issue to the content author to investigate further and update the document as appropriate.

sunnyhay commented 2 years ago

Thank you for the response. I'm looking forward to either building an online endpoint under which several deployments are put in different regions for failover consideration, or building several online endpoints deployed in different regions and routed under the same load balancer. Please advise the best practice or solution. For latter case, my user case is to route http requests among two URLs: https://pbi-endpoint.eastus2.inference.ml.azure.com/score and https://pa-endpoint.eastus2.inference.ml.azure.com/score. I expect to have a load balancer to route traffic between these two AML online endpoints.

sunnyhay commented 2 years ago

@YutongTie-MSFT any update so far? Thanks.