search

Azure / Azure-DataFactory

Other
481 stars 585 forks source link

PowerShell Script produced by azure-data-factory-utilities doesn't account for parameterised runtime state of triggers #387

Open crystalgeek opened 2 years ago

crystalgeek commented 2 years ago

I am having a problem with the recommended CI/CD for ADF. Using the azure-data-factory-utilities package.

When ADF trigger runtimestate is included in the arm-template-parameters-definition.json file "Microsoft.DataFactory/factories/triggers": { "properties": { "runtimeState": "=",

This produces an ARMTemplate which adds parameters to the runtimeState property ARMTemplateForFactory.json { "name": "[concat(parameters('factoryName'), '/BlobTrigger')]", "type": "Microsoft.DataFactory/factories/triggers", "apiVersion": "2018-06-01", "properties": { "annotations": [], "runtimeState": "[parameters('BlobTrigger_properties_runtimeState')]",

The powershell script for Pre/Post Deployment PrePostDeploymentScript.ps1 is expecting the runtimeState to be "Started" when defining triggers to start. This means that if parameterised these triggers don't get started even if they were previously

$triggersToStart = $triggersInTemplate | Where-Object { $_.properties.runtimeState -eq "Started" -and ($_.properties.pipelines.Count -gt 0 -or $_.properties.pipeline.pipelineReference -ne $null)} | ForEach-Object { New-Object PSObject -Property @{

Hao-Microsoft commented 2 years ago

Which azure-data-factory-utilities package are you using? Please share the location of the package.

crystalgeek commented 2 years ago

@.***/azure-data-factory-utilities

This npm package as described here

https://docs.microsoft.com/en-gb/azure/data-factory/continuous-integration-delivery-improvements#continuous-deployment-improvements

On Wed, 25 May 2022, 6:56 pm Hao-Microsoft, @.***> wrote:

Which azure-data-factory-utilities package are you using? Please share the location of the package.

— Reply to this email directly, view it on GitHub https://github.com/Azure/Azure-DataFactory/issues/387#issuecomment-1137646575, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB63SL3ILNMTAYOR6CUH4E3VLZSUDANCNFSM5WIJIURA . You are receiving this because you authored the thread.Message ID: @.***>

crystalgeek commented 2 years ago

@microsoft/azure-data-factory-utilities

This npm package as described here

https://docs.microsoft.com/en-gb/azure/data-factory/continuous-integration-delivery-improvements#continuous-deployment-improvements

dsfrederic commented 2 years ago

I'm facing the same issue. @Hao-Microsoft the script should be updated to take parameters into account

Hao-Microsoft commented 2 years ago

@crystalgeek and @dsfrederic , we are aware of this issue and will be working on the fix.

dsfrederic commented 2 years ago

In PR #433 I've provided a fix

Hao-Microsoft commented 2 years ago

Awesome thank you @dsfrederic ! It looks like you are already using the newer/better scripts. We'll fix this issue for both.

soma-ms commented 1 year ago

The issue has been fixed now in both places - in this repository (PrePostDeploymentScript.Ver2.ps1), and in npm package (version 1.0.0). If you are using NPM package, the fix is available only in the latest version and should use build-preview command as: https://learn.microsoft.com/en-us/azure/data-factory/continuous-integration-delivery-improvements#create-an-azure-pipeline