Azure / Azure-DataFactory

Other
483 stars 588 forks source link

Failed to compare payload for 'NNNNN' trigger Exception calling "Deserialize" with "3" argument(s): "The JSON value could not be converted to Microsoft.Azure.Management.DataFactory.Models.ScheduleTriggerRecurrence #471

Closed jimmyhjeltecab closed 1 year ago

jimmyhjeltecab commented 2 years ago

We have a problem with the new deployment model using the Ver2 script, all running triggers are stopped but never started and when I check the log this is what we see:

` Formatted command: . '/home/vsts/work/1/ArmTemplates/PrePostDeploymentScript.ps1' -armTemplate "/home/vsts/work/1/ArmTemplates/ARMTemplateForFactory.json" -armTemplateParameters "/home/vsts/work/1/ArmTemplates/ARMTemplateParametersForFactory.json" -ResourceGroupName "XXXXXXX" -DataFactoryName "XXXXXXXXX" -predeployment $true -deleteDeployment $false /usr/bin/pwsh -NoLogo -NoProfile -NonInteractive -ExecutionPolicy Unrestricted -Command . '/home/vsts/work/_temp/c98619c9-56f4-435d-b04a-a17cdfe86542.ps1' Saved! Import-Module -Name /usr/share/az_7.5.0/Az.Accounts/2.9.1/Az.Accounts.psd1 -Global Clear-AzContext -Scope Process Clear-AzContext -Scope CurrentUser -Force -ErrorAction SilentlyContinue Connect-AzAccount -ServicePrincipal -Tenant bd402493-0717-4f89-a565-39bdca08227b -Credential System.Management.Automation.PSCredential -Environment AzureCloud @processScope Set-AzContext -SubscriptionId b2d347d8-b915-47a9-889b-5e894fd11036 -TenantId bd402493-0717-4f89-a565-39bdca08227b Getting triggers

[warning] Failed to compare payload for 'XXXXXXXXX' trigger

[error] Exception calling "Deserialize" with "3" argument(s): "The JSON value could not be converted to Microsoft.Azure.Management.DataFactory.Models.ScheduleTriggerRecurrence. Path: $.recurrence | LineNumber: 18 | BytePositionInLine: 123." from Line: 204

[warning] Failed to compare payload for 'XXXXXXXXX' trigger

[error] Exception calling "Deserialize" with "3" argument(s): "The JSON value could not be converted to Microsoft.Azure.Management.DataFactory.Models.ScheduleTriggerRecurrence. Path: $.recurrence | LineNumber: 17 | BytePositionInLine: 123." from Line: 204

`

For example here is the code for one of our triggers { "name": "XXXXXXX", "properties": { "description": "Scale down XXX to 2 core", "annotations": [], "runtimeState": "Started", "pipelines": [ { "pipelineReference": { "referenceName": "XXXXXXX", "type": "PipelineReference" }, "parameters": { "Retries": 3, "ScaleRequestBody": { "sku": { "name": "HS_Gen5", "tier": "Hyperscale", "family": "Gen5", "capacity": 2 } }, "WaitTimeInSeconds": 300 } } ], "type": "ScheduleTrigger", "typeProperties": { "recurrence": { "frequency": "Week", "interval": 1, "startTime": "2022-07-05T06:18:00", "timeZone": "W. Europe Standard Time", "schedule": { "minutes": [ 0 ], "hours": [ 18 ], "weekDays": [ "Monday", "Tuesday", "Wednesday" ] } } } } }

soma-ms commented 1 year ago

Could you share the logs for script execution before/after the arm-template deployment ?

jimmyhjeltecab commented 1 year ago

@soma-ms I completly missed that you had replyed here are the logs

PreScript.log PostScript.log

We use a arm-template-parameters-definition.json and that one has the following code in regards to triggers, can that be the reason that you script can't parse our arm?

"Microsoft.DataFactory/factories/triggers": { "properties": { "pipelines": [ { "parameters": { "*": "=" } }, "pipelineReference.referenceName" ], "typeProperties": { "recurrence": "=:-recurrence:object", "maxConcurrency": "=" } } },

soma-ms commented 1 year ago

@jimmyhjeltecab - As you have custom parameterization for triggers, the script is unable to deserialize the trigger payload properly, so all the triggers are stopped before deployment. It seems the PostScript didn't execute at all and failed due to invalid value passed to script argument "CustomTriggersToStart". The original script doesn't have this argument, probably you customized the script. You may need to fix how you pass the values to this argument.