[Proposal] Allow infrastructure and services to be deployed in interdependent stages

[rjygraham]

There may be scenarios in which Azure infrastructure to be deployed as part of the "infrastructure" phase requires the deployment of one of the services defined in the azure.yaml; therefore, I propose azd allow developers to define multiple stages of infrastructure and service deployment within azure.yaml.

A real-world example of this requirement is establishing EventGrid topic subscriptions to an httpTrigger or eventGridTrigger hosted in Azure Functions. The function needs to exist at the time the EventGrid topic subscription resource is deployed, otherwise the subscription validation fails which results in a deployment failure.

Back of the napkin design for an updated azure.yaml supporting such functionality:

name: sample
infra:
  main:
    module: ./infra/main
  second:
    module: ./infra/second
    dependsOn:
      - infra/main
      - services/api
services:
  api:
    project: ./src/api
    language: py
    host: function
    dependsOn:
      - infra/main

The downside of this approach is that it adds unnecessary bloat to azure.yaml for less complex scenarios, so I believe it'd be useful to leave the infra section and dependsOn property optional for those scenarios.

[weikanglim]

Thanks for the suggestion. We'll definitely need some way to support this in azd to enable things like event grid subscriptions.

I could think of an alternative that is not as flexible, but perhaps more concrete to the scenario is to introduce pre-deploy and post-deploy hooks for services.

Something like:

services:
  api:
    project: ./src/api
    language: py
    host: function
    preDeploy:
       type: bicep
       module: ./infra/pre-deploy
    postDeploy:
       type: bicep
       module: ./infra/post-deploy

I think this would also allow additions like pre-deploy health checks and post-deploy operations.

[ellismg]

Something we should keep in mind while building this - I think we want to continue to be able to say something like: "give me all the resources that make up this application". We are already building editor experiences on top of the answers to this sort of question as well as commands like azd monitor use this list as part of their implementation. Today the way we answer this question is that we look at the deployment object for the main module and crawl that to discover resources. As we support multiple modules, we'll need to make sure we still know how to answer that question.

It is interesting that if you squint, this sort of looks like the problem we faced in container apps when we moved to deploying via bicep instead of using the container service control plane to directly to change the container definition that was running. In that world, we have to build and push the container image before creating the container app object but pushing the container image required the container registry which would not be provisioned until later.

I think if we come up with a suitable solution to the event grid problem it should translate here as well (and may be much nicer than the existing situation we have)

[TsuyoshiUshio]

azd is great and want to use it, however, this issue blocks me to using it. I'd like to +1 to the issue owner.