Bicep Parameters Upcoming Features

stephaniezyen commented 1 year ago

Main Story: #7301

Upcoming Features

Here are our upcoming features for Bicep Parameters. Please provide feedback and use this issue to mention any other features you would like included.

Supplemental Parameters in Command Line (Large)

[x] #11444 Use a command to validate params against a bicep file
[x] #11445 Update Az CLI logic to incorporate validate command
[x] #11446 Update Azure Pwsh logic to incorporate validate command

Modular Parameters (Large/Uncertain)

[x] #12019 I can import parameter files into others with an import directive
[x] #12019 I can pass values to the parameters in other files to enable replacement similar to Bicep modules

Other Tasks

[x] #10494 I can directly reference registry modules from bicep params files. (Medium)
[x] #11226 I can define variables in the parameter file using the variable declaration syntax in Bicep (Medium)
[x] #10652 add getSecret() support
[x] #10386 Support all parameters for User-Defined Types with intellisense
[x] #10978 Support command decompile-params and update command generate-params in AzCli
[x] #9971 Support command build-params in AzCli

alex-frankel commented 1 year ago

As it relates to passing parameters on the command-line, adding @anthony-c-martin's gist for implementation options: https://gist.github.com/anthony-c-martin/e2a605aca6218b8980c737163dfb7aeb

If we think the proposal is reasonable, I think it's important we get this one prioritized.

pchettri3 commented 1 year ago

Any update on June 1 release of v0.18 and the support for parameter file?

brwilkinson commented 1 year ago

dynamic parameters in powershell

I spent a bit more time testing bicepparam and it currently works as expected for Powershell, with the implementation that is part of Bicep CLI version 0.18.4

Example

using '../../bicep/00-ALL-SUB.bicep'
param global = {}

New-AzResourceGroupDeployment -global @{test = 123}

i.e. the above works for me, global value is overridden from the commandline.

This breaks with symbolicNameCodegen however that is covered in this other issue.

https://github.com/Azure/bicep/issues/10123

I am not sure if this always worked, I thought it was blocked for some reason previously.

With the implementation related to Environment vars, hopefully this continues to function.

I guess some people are not wanting to put any default value in the param file for global, however I am okay with some kind of default or placeholder, so long as it can be overwritten on the command line or references and environment variable optionally.

Usman0111 commented 1 year ago

dynamic parameters in powershell

I spent a bit more time testing bicepparam and it currently works as expected for Powershell, with the implementation that is part of Bicep CLI version 0.18.4

Example
using '../../bicep/00-ALL-SUB.bicep'
param global = {}
New-AzResourceGroupDeployment -global @{test = 123}
i.e. the above works for me, global value is overridden from the commandline.

This breaks with symbolicNameCodegen however that is covered in this other issue.

New-AzResourceGroupDeployment dynamic parameters cannot be used when deploying symbolic name templates #10123

I am not sure if this always worked, I thought it was blocked for some reason previously.

With the implementation related to Environment vars, hopefully this continues to function.

I guess some people are not wanting to put any default value in the param file for global, however I am okay with some kind of default or placeholder, so long as it can be overwritten on the command line or references and environment variable optionally.

Yes, it is supposed to be blocked but looks like this was missed in Powershell specifically

brwilkinson commented 1 year ago

Yes, it is supposed to be blocked but looks like this was missed in Powershell specifically

Got it, okay thank you. The current functionality works well for me 🙂

brwilkinson commented 1 year ago

I was assuming that I would have to make a change to my current format to the new format as below?

I think we can call this a breaking change, that would be part of the transition to bicepparam
plus it has the relatively low cost fix as below.

Example - Current

using '../../bicep/00-ALL-SUB.bicep'
param global = {}

New-AzResourceGroupDeployment -global @{test = 123}

Example - New format must refer to the specific env var to pass on commandline ?

using '../../bicep/00-ALL-SUB.bicep'
param global = readEnvVar('global')

New-AzResourceGroupDeployment -global @{test = 123}

Usman0111 commented 1 year ago

I was assuming that I would have to make a change to my current format to the new format as below?

Yes, that was the format we wanted for users to adopt as you are able to enforce some type safety though readEnvVar

chgennar commented 1 year ago

Related to "Modular Parameters", how would it work? Something like the following?

import globalConfig from '../global.bicepparam'
import otherCommonConfig from '../common.bicepparam'

param importedParam = globalConfig.paramToImport
param otherImportedParam = otherCommonConfig.otherToImport

I have some scenarios in mind where common configuration is stored across multiple files (e.g., globalConfig, siteConfig, useCaseSpecificConfig, etc.). I would like to be able to store all common configuration / parameter values in particular files, and then reference those files from the param file that will be used by the deployment.

ggirard07 commented 12 months ago

Using dotenv files for defining default values for `readEnvironmentVariable` during development/local usage

I have a use case where I initialize secrets in keyvault as part of bicep template. I used to specify those values from CLI. As I migrate to .bicepparam instead, I can now use readEnvironmentVariable directly to avoid specifying each in CLI command.

But this solution still fell short for development/local story. I have error reported as those environment variable do not exists on my local machine. In this case, I want to use some dummy values instead of actual secrets. I cannot put those directly as default value for readEnvironmentVariable as I do not want them in production. Having the possibility to override my local environment from some .env file instead seems to be really appealing Even if this is secrets I do not care to push file to repo as this is dummy values for dev purpose only. Only problem I am not sure about is that those files should be loaded by the VS Code extension only, not by az CLI, otherwise I am back to the same issue as putting default values inside .bicepparam directly.

What are your tips about defining default values locally to avoid Environment variable does not exist, and no default value set bicep(BCP338)?

jordanrbaker commented 12 months ago

Related to #11319 and @ggirard07 's comment

If process scope environment variables are intended to be unsupported - suggest updating docs (https://learn.microsoft.com/en-us/azure/azure-resource-manager/bicep/bicep-functions-parameters-file#readenvironmentvariable) to explicitly state this.

Rationale: Environment variables are tricky therefore this will likely be a common linting error. Explicitly stating as much as in the documents will drive bicepparams adoption by reducing pain points in migration from JSON.

jikuja commented 12 months ago

If process scope environment variables are intended to be unsupported

What do you mean by process scope environment?

What are your tips about defining default values locally to avoid Environment variable does not exist, and no default value set bicep(BCP338)?

TBH VS code (extensions) has really bad developer experience to add environment variables or read environment variables from file for tooling launched directly from the VS code. It might be easier to use terminal, read enviroment file and use CLI tooling.

jordanrbaker commented 12 months ago

If process scope environment variables are intended to be unsupported

What do you mean by process scope environment?

What are your tips about defining default values locally to avoid Environment variable does not exist, and no default value set bicep(BCP338)?

TBH VS code (extensions) has really bad developer experience to add environment variables or read environment variables from file for tooling launched directly from the VS code. It might be easier to use terminal, read enviroment file and use CLI tooling.

My issue was that when ingesting variables on external runners they were by default set to process, but needed to be set to at least user to work. This isn't an issue per se but it may trip up some newer users.

It's also possible this behavior is not intended.

[System.Environment]::SetEnvironmentVariable('TESTVAR','TEST', 0) - Process, does not work, variable can't be read by linter or when compiling a template directly at the CLI.

Error BCP338: Failed to evaluate parameter "TESTVAR": Environment variable does not exist, and no default value set

[System.Environment]::SetEnvironmentVariable('TESTVAR','TEST', 1) - User, works as expected. [System.Environment]::SetEnvironmentVariable('TESTVAR','TEST', 2) - Machine, works as expected.

https://learn.microsoft.com/en-us/dotnet/api/system.environmentvariabletarget?view=net-7.0

jikuja commented 12 months ago

Weird. How do you invoke Bicep?

Result on my machine:

bicep --version
Bicep CLI version 0.18.4 (1620479ac6)

other cosmetic issue:

related with my earlier comment about tooling and VS code

brwilkinson commented 12 months ago

@jikuja latest release is 0.19.x so try update bicep cli.

jikuja commented 12 months ago

@jikuja latest release is 0.19.x so try update bicep cli

Same results. Works for me, no regression.

brwilkinson commented 12 months ago

@jikuja oh, I see, it is working at expected.

Sorry I missed context before. I thought I was not working.

About having the Env var set for the Bicep vscode extension and having the process pickup newly set values.

I think supporting the (dotenv) .env file method, which was suggested above, might be the cleanest/easiest solution? Your local testing. Which is part of .gitignore and not checked in.

Mantene commented 10 months ago

Regarding "I can import parameter files into others with an import directive", this would be super useful for things like the Application Gateway where you are adding new backend applications as time goes on, especially if it can merge parameters with the same names, such as probes:

param probes = [
      {
        name: 'Tiger-Speed-API-Probe'
        id: '${componentIdPredicate}/probes/Tiger-Speed-API-Probe'
        properties: {
          protocol: 'Https'
          path: '/api'
          interval: 300
          timeout: 60
          unhealthyThreshold: 3
          pickHostNameFromBackendHttpSettings: true
          minServers: 0
          match: {
            statusCodes: [
              '200-404'
            ]
          }
        }
      }
      {
        name: 'Tiger-Speed-UI-Probe'
        id: '${componentIdPredicate}/probes/Tiger-Speed-UI-Probe'
        properties: {
          protocol: 'Https'
          path: '/'
          interval: 30
          timeout: 20
          unhealthyThreshold: 1
          pickHostNameFromBackendHttpSettings: true
          minServers: 0
          match: {
            statusCodes: [
              '200-404'
            ]
          }
        }
      }
    ]

So I could have a separate bicepparam file for each application that needs to be behind the Application Gateway, making maintenance so much more organized.

ggirard07 commented 9 months ago

Is it possible to include #10454 in this scope?

quality-leftovers commented 2 months ago

anthony-c-martin commented 2 months ago

@stephaniezyen since the only remaining item is the modular parameters feature, should we close this issue and track that under a more specific issue?

Azure / bicep