Azure / Azure-DataFactory

Other
481 stars 585 forks source link

Factory setting - Show billing report value being lost after deployment #547

Open Bhuard1 opened 1 year ago

Bhuard1 commented 1 year ago

Hi

We activate the public preview feature Show billing report Blog: https://techcommunity.microsoft.com/t5/azure-data-factory-blog/granular-billing-for-azure-data-factory/ba-p/3654600 (the blog states that is not part of ci/cd process)

We still use the GlobalParametersUpdateScript.ps1 script to update global parameters Even tough the setting not part of the global parameter file, running this script will set back the value to 'by factory'

(in git Show billing report - By pipeline appears as "globalConfigurations": {"PipelineBillingEnabled": "true"} So running the script (uses 'system.collections.generic.dictionary[string,Microsoft.Azure.Management.DataFactory.Models.GlobalParameterResource]) must remove the globalconfiguration value

Attached script if it helps GlobalParametersUpdateScript.zip

sticl commented 1 year ago

Hi, we're experiencing the same issue here. It seems there is no easy way to override these parameters across environments in the new CI/CD flow.

It also doesn't remember that it's set to 'by factory' across environments, so it doesn't seem like an embedded setting either. It would be good to also allow these to be overridden in here, via overrideParameters:

image

Same way like you can override global parameters and such...

soma-ms commented 1 year ago

@Bhuard1, @stijncl12 - I think the globalConfiguration property is getting overridden with the PowerShell script. I would suggest using new mechanism to include global parameters in CICD which is a recommended way. And you won't run into this issue there.

sticl commented 1 year ago

@soma-ms I'm not using the global parameters powershell script at all in our pipeline, and I am using the new mechanism.

image

The problem is that if I export the arm template from within the portal or when CI/CD does the same thing, this setting is nowhere to be found in the files ARMTemplateForFactory.json & ARMTemplateParametersForFactory.json.

This property seems to be part of globalConfigurations rather than globalparameters. Normally this should cause it to be exported as it's defined in the parameter configuration:,

image

But this doesn't seem to have any impact at all, causing it not to be parameterizable across environments as we do with global parameters. (by the way, this parameter configuration was in there by default)

I guess this is a duplicate of this issue: https://github.com/Azure-Player/azure.datafactory.tools/issues/241

Bhuard1 commented 1 year ago

@soma-ms Hi, I must admit that these types of responses are disappointing. The 'new way' is even harder to deploy as we have to put more variables in the release of the ARM template. Having a config file per environment much easier to manage in Git. The powershell file is to be supported (as documented in blog/and MS documentation), difficult to understand why you don't want your feature to be used in the your documented CI/CD process. In our dev Live mode we have simply disabled the global parameter update in the CID/CD process. We will not be able to use the feature in our other environnement until this is adressed.