Azure / Azure-DataFactory

Other
481 stars 586 forks source link

CI/CD Script does not detect changes in parameter values in object arrays #542

Open aurelschwitter opened 1 year ago

aurelschwitter commented 1 year ago

Example trigger config:

{
    "name": "schedule_daily_execution",
    "properties": {
        "annotations": [],
        "runtimeState": "Started",
        "pipelines": [
            {
                "pipelineReference": {
                    "referenceName": "my_pipeline",
                    "type": "PipelineReference"
                },
                "parameters": {
                    "tableConfig": [
                        {
                            "tableName": "table1"
                        },
                        {
                            "tableName": "table2"
                        }
                    ]
                }
            }
        ],
        "type": "ScheduleTrigger",
        "typeProperties": {
            "recurrence": {
                "frequency": "Day",
                "interval": 1,
                "startTime": "2022-09-21T09:32:00Z",
                "timeZone": "UTC",
                "schedule": {
                    "minutes": [
                        30
                    ],
                    "hours": [
                        4
                    ]
                }
            }
        }
    }
}

When changing the table names to "table3" and "table4", the script does not detect a change because compare-object does not compare array contents:

pwsh > $oldConfig = '[{"tableName":"table1"},{"tableName":"table2"}]'
pwsh > $newConfig = '[{"tableName":"table3"},{"tableName":"table4"}]'
pwsh > Compare-Object (ConvertFrom-Json $oldConfig) (ConvertFrom-Json $newConfig)
# no output --> no change
soma-ms commented 1 year ago

@aurelschwitter - The script will do best-effort comparison to detect changes in trigger payload. You can pass the trigger list to ExplicitStopTriggerList argument to explicitly stop them as the deployment fail if the trigger is not stopped.

We will investigate and check if the issue can be addressed.

himanshu-ram-punjabi commented 4 months ago

I started facing the same issue today. All my Triggers now have a Object Parameter (due to limitation of 256 Parameters). While I can use ExplicitStopTriggerList to stop the Triggers, there is no way to Start all of them after the Deployment.

Please see if a solution can be proposed. Thank you.