Azure / bicep

Bicep is a declarative language for describing and deploying Azure resources
MIT License
3.21k stars 745 forks source link

Paramter files - Allow creation of parameters file without using #10333

Open slavizh opened 1 year ago

slavizh commented 1 year ago

Is your feature request related to a problem? Please describe. Seems using declaration is required. This assumes that the parameters file is in the same location as the bicep templates. In CI/CD scenarios where the templates are located in storage like Azure DevOps artifacts the templates are not in the same repository/location as the parameters file. They are not and when you do CI/CD the template is picked from Azure DevOps artifact and executed with the parameters file on the pipeline run. So for cases where you want to have the bicep syntax for parameters file but you do not want to tie it to specific template.

Additionally in the future you should be able to use 'using' declaration against bicep files that defines user defined types rather bicep template. Also you should be able to use 'using' against file located in web rather locally stored ones so when you deal with multiple templates and multiple versions of those you do not have to store every template locally instead you just reference url where the main.bicep file is located or file holding user defined types.

Describe the solution you'd like Described above.

alex-frankel commented 1 year ago

This assumes that the parameters file is in the same location as the bicep templates

Our assumption was that specifically during the development process, these files would be co-located. Are you saying that you will develop a bicepparam file and expect to be able to reference a file in remote storage? Can you give some more specific timelines of when bicep code is being written relative to bicepparam code and the sequence of events in getting this code checked-in/validated.

Additionally in the future you should be able to use 'using' declaration against bicep files that defines user defined types rather bicep template.

Can you clarify this a bit more? I am not sure what this is referring to.

FWIW, we expected that it may not be possible to make using mandatory, but we wanted to start strict and then relax the constraint as needed.

slavizh commented 1 year ago

Ok, let me try to explain it. Let's start with user defined types. One of the features of user defined types is to be able to verify if the input parameters are correct based on your user defined types definition. As far as I understand it in the end user defined types is just converted to something similar to json schema. Let's take the case we have multiple bicep templates that represent a single solution that deploys on or more resources. For the parameters you want to defined user defined types so you can verify if the input is correct. It will be easier and allows for better development and modularity the user defined types to be in separate file. In the main bicep file you just import the file with the user defined types so you can use them in it. An end user who will deploy this solution will be creating bicep parameters file. It will be a lot easier if in their parameters file they reference the file with the user defined types either locally or on public storage endpoint. That way they do not have to download all the templates of the solution just to create bicep parameters file with the correct configuration. The download of the bicep templates can happen during pipeline run. You can have a task that does that. To put this additionally to perspective the bicep templates (solution) can be versioned with semantic versioning. Let's say you have released version 1.0.0, Bicep parameters file was created against the user defined types available for version 1.0.0. A new version 1.1.0 is released. When new version is released you publish the bicep templates to package tool like Azure Artifacts in a new version, you publish the user defined type as a new version (for example on storage account within folder named 1.1.0). In your bicep parameters file you just have to reference the user defined type for version 1.0.0 and for example now you see that you have a new feature that consists of one or more new parameters that you can configure in your bicep parameters file.

This whole process for me sounds very similar to how you write bicep code with referencing resource types and versions and you get the properties available. I think the bicep parameters file should offer the same experience where you reference the user defined types which describe what is possible to deploy with the template when you provide different input.

I hope this describes clearly what I have in mind and what I think would bring quite improvements on doing CI/CD with Bicep.

If we go with using in the current state of Bicep, basically and end user will have git repository with bicep parameters file and yml defined pipeline that starts bicep deployment. In order to do deployment that user will have to go and download the version of the solution that he/she will be using. Not only download but also commit those files to the repository. This sounds like very bad experience to me. Not to mention they could download the wrong version or in case they need to try a new version they need to download the new version and if they have some failure they will need to revert. With the described referencing they could just modify a few strings and files without having to download anything and store anything that is already stored as artifact or as Template spec somewhere.

slavizh commented 1 year ago

For anyone that is annoyed like me about this constrain you can put the only the parameters from your main.bicep file into another file and reference that file in your Bicep parameters file. That way if your Bicep deployment consists of multiple templates and not just a single main.bicep file you can avoid copying all these template files just to get intellisense in your parameters file. It would be really helpful and way better experience if using was supporting also http endpoint and not only local files.

slavizh commented 1 year ago

One additional scenario to make using non-mandatory. When you have template spec the template spec is located as resource in Azure. When you upload the template spec it is actually compiled from Bicep to ARM Template. The whole point of template spec was to be able to store these in central place and people to use them from that central place. In this kind of scenario not only you have to maintain local copy but that local copy needs to be Bicep file and the template spec will be ARM Template json, That is of course if somehow using is allowed to specify template spec resource ID but this means that using needs to support also the JSON ARM template and not only bicep. Other option is to point to template spec resource id using but bicep cli on the background de-compiles the JSON to Bicep. Of course not sure how good that would be as the JSON was compiled from Bicep by using certain Bicep version and if you de-compile with another versions there could be problems so it does not de-compiles correctly or at all.

All this leads to several suggestions I have made before not only for parameters but for other features:

BTW if you decompile Bicep parameters file to JSON and after that you compile back to Bicep parameters file the using value is lost. One scenario for doing this compilation/de-compilation is if you want to somehow inspect or modify the configuration on your own in PowerShell it is easier if you compile bicep to JSON, convert JSON to PowerShell object. In PowerShell it is just easier to work with PowerShell objects instead of text files and there is no commandlet to convert Bicep parameters file to PowerShell object. Once you have done your checks/modifications you can convert back to JSON and from there to Bicep parameters file via the bicep cli. But in that case the using statement is lost and that is mandatory.

feliasson commented 1 year ago

I want to be able to create the bicep parameter files without using as well.

Just to give an example. This is how I expected to be able to use the bicep parameter files (I have already been doing like this but with the parameter files in ARM/JSON):

// param-example.bicepparam

param location string = resourceGroup().location

param tags object = {}

param env object = {
  acr: {
    name: 'acr01'
    location: location
    acrSku: 'Standard'
    tags: tags
  }
  aks: {
    name: 'aks01'
    location: location
    loadBalancerSku: 'Standard'
    tags: tags
  }
  kv: {
    name: 'kv01'
    location: location
    tenantId: tenant().tenantId
    tags: tags
  }
}

When I call main.bicep, I just want to send in the whole env object and combine the inputs to the modules with such as the contains function.

// main.bicep

param location string = resourceGroup().location

param tags object = {}

param env object = {}

module acr 'modules/acr.bicep' = {
  name: 'deploy-acr-123'
  params: {
    name: env.aks.name
    location: contains(env.acr.location) ? env.acr.location : location
    acrSku: contains(env.acr.acrSku) ? env.acr.acrSku : 'Standard'
    tags: contains(env.acr.tags) ? env.acr.tags : tags
  }
}

module aks 'modules/aks.bicep' = {
  name: 'deploy-aks-123'
  params: {
    name: env.aks.name
    location: contains(env.aks.location) ? env.aks.location : location
    loadBalancerSku: contains(env.aks.loadBalancerSku) ? env.aks.loadBalancerSku : 'Standard'
    disableLocalAccounts: contains(env.aks.disableLocalAccounts) ? env.aks.disableLocalAccounts : true
    tags: contains(env.aks.tags) ? env.aks.tags : tags
  }
}

module kv 'modules/kv.bicep' = {
  name: 'deploy-kv-123'
  params: {
    name: env.kv.name
    location: contains(env.kv.location) ? env.kv.location : location
    tenantId: contains(env.kv.tenantId) ? env.kv.tenantId : tenant().tenantId
    tags: contains(env.aks.tags) ? env.aks.tags : tags
  }
}

I hope my comment does not interfere with the already started discussion.

slavizh commented 9 months ago

From the community call - I vote for Scenario 2.

brwilkinson commented 9 months ago

Can we confirm. With PowerShell right now, there is no checking on the using?

i.e. Currently you can deploy bicepparam with any bicep file.

Please (🙏) don't change this. This would be a major breaking change for me at this point.

I believe this aligns with option/scenario #2... Which I assume impacts az cli. To bring this in alignment with Powershell.

ericyew commented 7 months ago

same issue here. We standardised our template pattern and store them in a different repos to the parameter file so they are not in the same folders. Will keep using json parameter file until this gets resolved.