Open savannahostrowski opened 1 year ago
I would lean heavily into our CI providers: GitHub and Azure Pipelines, and provide a mechanism that allows azd
environments to easily be reused in those scenarios. I would also think about how we can build azd gestures that solve common problems around environment promotion, while ensuring that local workflow matches CI workflow as much as possible.
@savannahostrowski @ellismg This likely should be required as part of our ADE integration
What is our guidance for managing that? How do we enable a developer to advance their app from one environment to another (e.g. dev -> prod)?
Our current strategy allows you to set per environment configuration for your IAC, so at a minimum you could imagine adding a new parameter to your infrastructure on what sort of environment type (dev, prod, etc) you were deploying to and your infrastructure template could conditionalize resources on that.
Or, you could just examine AZURE_ENV_NAME
in your bicep and based on it's structure do things (perhaps your organization has a way to name environments that encodes some semantics).
I think that can get us very far today. To @weikanglim's point, you can define these values in your CI system (perhaps tied to branches, perhaps now).
It would be good to know where this falls over or where the rough edges are and we can start to sand them down.
@savannahostrowski @ellismg This likely should be required as part of our ADE integration
This sort of surprised me - given that some of the initial design work in the ADE stream led me to believe that in ADE mode we'd only allow a single environment per project. What do you think is going to push us to figure this out for ADE, @wbreza?
This sort of surprised me - given that some of the initial design work in the ADE stream led me to believe that in ADE mode we'd only allow a single environment per project. What do you think is going to push us to figure this out for ADE, @wbreza?
ADE is all about separating out the governance and creation of infrastructure typically required in a more structured organization or enterprise. Devs won't have access to provision resources, but if the enterprise wants to leverage ADE + AZD for integration scenarios we should have a good story here.
ADE devs will likely only be working directly against lower level environment types like "dev" but outside of the dev focused scenario test/staging & prod environments still exist and should be supported in CI/CD scenarios.
Most of this boils down to correct environment configuration in Azdo or Github but once that has been setup azd should be able to easily leverage this configuration where calls to azd provision
link to the correct ADE environments across the full phase of deployment rollouts.
Proposal and use cases for multiple environment support in azd
.
Today azd is limited to 1 environment definition via azd pipeline config
command.
Both Github & Azdo support the concept of "environments". The concept maps directly towards environments in azd
. It is a separation of configuration + azure resources that support deploying directly into each of these environments.
Note Github environment are only available for public repos. Private repos require having a paid version of GitHub pro / enterprise
Today, azd
creates repo level variables and secrets in Github and pipeline variables in Azdo.
--ci-environment
flag to target which CI provider environment to use. Default convention would be to map to the same environment name as azd. azd
creates this new GitHub environment if it does not already existazd
creates all secrets / variables within this environmentWarning If environments aren't available for current repo default to single environment support?
azd
creates new AzDo environment if it does not existazd
creates a new variable group for the environment (use environment name by default)azd
creates new secrets & variables within this variable groupQ: I want to change the infrastructure that is provisioned per environment
A: For simple changes like supporting availability zones, different service SKUs, scaling settings users should leverage IaC template parameters to drive different deployment paths based on 1 or many input parameters. If param mode=dev
use lower end skus. If param mode=prod
use higher end skus.
Q: I want to use different service targets per environment. Container apps in dev, AKS in prod A: This one can't be handled in pure IaC templates because the packaging and deployment needs to change per environment.
In this case users can author multiple versions of the azure.yaml
. Base configuration can still exist in azure.yaml
. azd
will overlay additional changes for azure.{env}.yaml
during project loading. Only deltas needs to be supplied in the environment specific version.
This provides the ability to swap out any configuration element per environment. IaC template paths, IaC providers and even services if needed.
azure.yaml
azure.{env}.yaml
For app code promotion we typically want to take the exact version deployed from a lower environment and promote/release it to an upper environment. This is important because this is the exact version that was tested and known to be in a good working state vs just re-building from your mainline branch.
The simpler case relies less on Azd and more on writing the correct multi-stage pipelines in your CI provider of choice. This approach will do a direct code deployment to your configured service target.
azd run package <service>
to generate the deployable artifacts for your services
azd deploy <service> --from-package <package-path>
referencing the appropriate artifactService Targets
swap
capabilityAdd a --rollout
or similar param to azd deploy
to perform service target specific rollout of application over a period of time
...
Overall, I love the proposals here and think they are a step forward in the right direction to having azd
be a great way to manage Azure in CI/CD workflow which has multiple deployments of the application (production, staging, testing, etc)
A few comments based on what popped into my mind while reading.
If environments aren't available for current repo default to single environment support?
Yeah, this seems like about the best we can do. It is interesting, I think, that when we create a repository during pipeline config
we create a private repository by default (IIRC), which wouldn't support this feature. Not sure how to think about that or if we should care.
Q: I want to use different service targets per environment. Container apps in dev, AKS in prod A: This one can't be handled in pure IaC templates because the packaging and deployment needs to change per environment.
No opposed to a azure.<env-name>.yaml
that overrides stuff from azure.yaml
. Did want to note that I wonder if we could relax/remove the need to put the service type in azure.yaml
at all or if long term we would want to try to detect this based on the target resource we discover for the service (based on tags, or some other strategy, like the infrastructure having exports of the form SERVICE_<name>_RESOURCE_ID
or something).
I guess I also wonder in practice if anyone would actually do this. It feels sort of like an anti-pattern to have such a large change in the way your application is deployed between environments, not really sure how common such a thing would end up being in practice?
Run azd run package
to generate the deployable artifacts for your services We need better JSON output version to be able to programmatically get the artifact path from the package command.
I wonder if instead we should support something like azd package --package-path <path-to-file>
and in this case, azd package
just uses <path-to-file>
instead of generating a filename itself. The pipeline author could then place the artifact wherever they please. This may be easier than running azd package --output=json | jq .packagePath
or something.
Add a --rollout or similar param to azd deploy to perform service target specific rollout of application over a period of time
I do wonder again how much of this we could infer from the target resource. Would it possible to examine an AppService resource and see if it is using deployment slots and if so deploy that way? Similar for Container Apps, could we detect when the app is configured for single vs multi-revision mode and change our strategy based on that? There's also a question longer term if we'll want a way for someone to override what "deploy" means for a service to instead use some other deployment strategy (maybe something like spinnaker or some other deployment strategy that is not strictly tied to the service).
Is this the relevant issue to follow for deployment slots support for App Service?
A developer has asked how they can deploy to a slot using azd up: https://github.com/Azure-Samples/azure-search-openai-demo/issues/1278
I did some work around this and created this template and some documentation. These demonstrate how I'm using AZD, GHA, and Bicep to create and configure support for multiple environments. I presented this topic at the Azure User Group: https://www.youtube.com/watch?v=nSebeKFG2s8.
As far as GHA is concerned, support for environment variables and secrets is needed. For AzDO, I have yet to dive as deep, but support for environments and variable groups is required.
I hope this is helpful; I am happy to discuss this further.
Are there any updates on this issue? Using azd to fastrack a developer from idea to Cloud dev environment is good, but I think there needs to be better support to take this through environments into production using pipelines or github actions with environment scoped secrets and variables. Even if it was one good example included in Awesome AZD templates similar to what jasontaylordev has done above, or with the current best approach from the azd team.
You can have environment-scoped secrets and variables in your pipelines now, by the way. We use them in https://github.com/Azure-Samples/azure-search-openai-demo/ There's a pending PR that adds a section about them to the azd pipeline documentation, as you need to modify both azure.yaml and your pipeline files to know about the variables.
You can have environment-scoped secrets and variables in your pipelines now, by the way. We use them in https://github.com/Azure-Samples/azure-search-openai-demo/ There's a pending PR that adds a section about them to the azd pipeline documentation, as you need to modify both azure.yaml and your pipeline files to know about the variables.
I had a review of the documentation here; https://learn.microsoft.com/en-us/azure/developer/azure-developer-cli/configure-devops-pipeline?tabs=GitHub#additional-secrets-or-variables.
This new feature seems to allow us to specify additional variables or secrets to include when running azd pipeline config
. Regarding environment-scoped, the values will come from the current azd environment. If using GitHub as the pipeline provider, the necessary secrets and variables will be created, but they will be scoped to the repository rather than a specific GH environment.
This seems like a good feature, but it doesn't bring me closer to configuring multi-environment support for a GH pipeline.
I would like the azd environment to be mapped directly to the pipeline environment. This way I can create variables and secrets at the GH environment scope. Ideally, I would also like some variables or secrets to be repository-scoped. Then, it's just a case of creating a multi-stage pipeline to promote releases between environments.
Developers may want different IaC for each environment (dev, test, prod).
Note that using branches for management is sort of an anti-pattern/dated approach and there are developers who do not view this as an option (e.g. https://codefresh.io/blog/stop-using-branches-deploying-different-gitops-environments/)