Azure-Player / azure.datafactory.tools

Tools for deploying Data Factory (v2) in Microsoft Azure
https://azureplayer.net/adftools
MIT License
209 stars 69 forks source link

Fixing config validation failure on booleans #273

Closed tctylerFC closed 1 year ago

tctylerFC commented 1 year ago

When running Test-AdfCode, if your deployment config files contain Booleans, the script will fail with the following error:

Validation of config file completed.
ERROR: The variable '$123' cannot be retrieved because it has not been set.

This occurs because the ObjectProperty.ps1 script receives "123" as the "$value" var when "$dryRun" is "$true". Setting "$value" = "$false" and correcting "$exp" so that it receives "$false" as its value instead of "$$false".

NowinskiK commented 1 year ago

The script receives what you configured in config.csv/json file. Because you didn't show it here, I assume you did it wrong. Probably in your config file you had value like 123 or $false (rather than false)

tctylerFC commented 1 year ago

Demo Config file deploying just an integration runtime for the sake of testing.

type,name,path,value integrationRuntime,DEMO-AZIR,typeProperties.computeProperties.dataFlowProperties.coreCount,"16" integrationRuntime,DEMO-AZIR,typeProperties.computeProperties.dataFlowProperties.computeType,"General" integrationRuntime,DEMO-AZIR,typeProperties.computeProperties.dataFlowProperties.cleanup,false integrationRuntime,DEMO-AZIR,typeProperties.computeProperties.copyComputeScaleProperties.dataIntegrationUnit,"8" integrationRuntime,DEMO-AZIR,typeProperties.computeProperties.pipelineExternalComputeScaleProperties.timeToLive,"30"

The code I'm referencing that leads to the failure when passed a boolean variable.: Starting on line 138 of Update-PropertiesFromFile, dryrun sets $value to 123, then calls Update-Object Property

if ($dryRun) { $value = 123 }
if ($validPath) {
    switch -Exact ($action)
    {
        'update'
        {
            Update-ObjectProperty -obj $json -path "properties.$path" -value "$value"
            $report.Updated += 1
        }
        'add'
        {
            Add-ObjectProperty -obj $json -path "properties.$path" -value "$value"
            $report.Added += 1
        }
        'remove'
        {
            Remove-ObjectProperty -obj $json -path "properties.$path"
            $report.Removed += 1
        }
    }
}

In ObjectProperty.ps1, line 14, the "$exp" var sets to $$value

elseif ($fieldType -eq [Boolean]) { Write-Debug "Setting as Boolean value" $exp = "$obj.$path = $$value" }

"$exp" becomes "$obj.$path = $123". While $obj.$path can be resolved, the variable $123 doesn't exist.

NowinskiK commented 1 year ago

If your target value is type of boolean, you must provide boolean value, not numeric.

tctylerFC commented 1 year ago

Maybe I'm still misunderstanding... so the script I was quoting where that numeric value ($value = 123) is being set is part of Azure.datafactory.tools. - azure.datafactory.tools/private/Update-PropertiesFromFile.ps1

tctylerFC commented 1 year ago

Is there something more I can provide to prove the need for this fix? Or is there an alternate fix you can suggest to correct the issue where providing a boolean variable in the configs causes a failure?

NowinskiK commented 1 year ago

Actually, the dryRun line shouldn't be there, so I need to remove it.

tctylerFC commented 1 year ago

Closed this PR per @NowinskiK's last comment as my proposed resolution will no longer be necessary if we remove the $dryRun line