Azure / data-factory-deploy-action

GitHub Action for side-effect free deployment of Azure Data Factory resources
MIT License
29 stars 27 forks source link

Question about active triggers #8

Open abrennan24 opened 2 years ago

abrennan24 commented 2 years ago

I've recently migrated to a GitHub repo to manage my data factory deployments and have started using this action. However I'm running into a scenario that doesn't work as I would expect.

When the pre-deployment script determines the "active" triggers, it's getting that information from the template and NOT from the target data factory:

$triggersToStart = $triggersInTemplate | Where-Object { $.properties.runtimeState -eq "Started" -and ($.properties.pipelines.Count -gt 0 -or $_.properties.pipeline.pipelineReference -ne $null)} ...

The template has the state of our DEV environment. When we're deploying to test and prod, we have different sets of triggers running. What I would expect to happen is that it would look at the triggers in the target data factory to find the ones that are active (and should therefore be restarted when deployment is completed).

This may be a simple fix to change to line 159 in the script:

$triggersToStart = $triggersInTemplate |

to be:

$triggersToStart = $triggersDeployed |

That would match how the triggersToStop and triggersToDelete are captured.

I'm not clear if that's the intended behavior or not.

[UPDATE]: I understand now why the one-line fix wouldn't work, as the post-deployment is a separate invocation of the script, so when looking at the triggers in the target data factory, they would already be stopped at that point.

I have created a custom version of this that will serialize the active triggers in the data factory during the pre-deployment phase, and it will use that list in the post-deployment phase to restart the triggers that were previously running. I could create a pull request with changes if that's something people would be interested in.

I'm curious if anyone else has run into this. Do people normally have the same triggers enabled in dev/test/prod environments? On my teams, we typically don't keep our dev and test environments continuously updating as we do in prod. Dev and test are generally more for running on-demand pipelines for debugging and testing.

sxs737 commented 1 year ago

@abrennan24 Do you have Fork we can test . We are facing similar issue with triggers

abrennan24 commented 1 year ago

@abrennan24 Do you have Fork we can test . We are facing similar issue with triggers

My changes are in an internal repo, but here's the modified deployment script. PrePostDeploymentScript.zip