Split container app deployment out from provisioning

[weikanglim]

Currently, azd deploys container apps (non-Aspire) by updating the container app properties in the main provisioning flow, then patching the container app image during azd deploy.

The primary pain point is that users do not have the ability to patch the container app in a single revision with the provision + deploy flow (#3116). A secondary pain point is that if the user doesn't store the container app existence state in CI/CD scenarios, the container app image may be reset during CI/CD provisioning. The UX is also slightly more challenging since multiple parts in the application have to be set up and coordinate well together to build up the overall experience.

There are two ways we can consider splitting out container app deployment from provisioning:

1. Container app deployed using Bicep
2. Container app deployed using YAML

Container app deployed using Bicep

This is familiar for a person that is using and knowing Bicep. It is also quite common to have multiple Bicep modules that are deployed and managed independently. Bicep allows the user to reference any existing provisioned resources and link it to fill application environment settings.

The main drawback from azd's perspective is that azd interacts with Bicep opaquely. azd doesn't know if the user had provisioned other resources within the Bicep module. azd can only communicate with the Bicep module via input/outputs just like other provisioning modules.

Historically, this deployment mechanism was previously used by azd. It was noted to be slow, however this may have changed recently.

Container app deployed using YAML

This is familiar for a person that has prior K8s experience. A manifest style approach for an app operator that doesn't need to touch infrastructure. This provides the right amount of separation for the app developer without having to learn IaC.

The expectation would then be that azd needs to provide and document templating functions to "tie-in" the container app to the rest of the application. At the very least, I could imagine:

• The context environment variables (especially a managed identity, if being used)
• The container app environment ID
• A way to reference KeyVault secrets
• A way to reference properties (perhaps $.jsonpath) of arbitrary ARM resources by resource ID

Discussion

Should container app deployment be split from provisioning?

Do you prefer deploying container app using Bicep or YAML?

[jongio]

I would be hesitant to complicate the ACA deployment scenario with another YAML spec.

Can you provide an example of:

1. Customer complaints/concerns on this
2. Demo of existing issues
3. Demo of proposed solution

Thanks

[wbreza]

Agree on the pain points listed above. It is a tricky situation since the Bicep (CAE + ACA) today is a full provision+deploy that can contain references to the container images being used by the service.

When your application container images aren't available yet then you won't have a final container image to use and pass to your bicep module to set the parameter(s).

When thinking about the separation of provisioning & deployment when it comes to ContainerApps, the ACA environment is the resource that theoretically could be provisioned, but the ACA apps themselves are not created until the deploy stage (similar to Aspire).

Then in the deploy stage we could then choose to either user bicep or yaml. Some customers may prefer bicep while other prefer yaml and maybe we need to support both.

In a bicep world the module can take many parameters used to set container image references as well as other configurable properties. Also - if azd is moving to deployment stacks, then each ACA deployment would need to be a separate stack and deployed independently. This opens the door for root level infra to be considered "platform" infra while each service can provide service level infra like ACA apps, etc.

In Yaml, I agree that we would need some text templating language to allow the injection of azd environment/configuration to form a complete well formed yaml document.

If I had to pick one path to start with I'd opt for CAE to be provisioned during standard provision stage and ACA apps to be created via YAML deployment via azd deploy stage.

[ellismg]

If I had to pick one path to start with I'd opt for CAE to be provisioned during standard provision stage and ACA apps to be created via YAML deployment via azd deploy stage.

This is what we did for Aspire. For .NET 9 we are changing the model such that instead of using YAML for deployment (which required us to talk to the container apps RP directly, and invent a templated yaml logic in the style of helm) to generate a bicep file for just the container app which we will deploy using the ARM control plane, during the deploy phase.

Work for that is being prototyped in #4286.

We moved away from YAML and towards Bicep because Aspire is working with the .NET Team to produce a CDK which makes it easy to generate Bicep using C# and we felt this would end up being more natural for users long term instead of yet another IaC format designed specifically by azd.