search

Azure-Player / azure.datafactory.devops

Azure DevOps Deployment Tasks for Azure Data Factory objects
MIT License
39 stars 21 forks source link

Delete objects not in source - functionality #94

Closed nimnp6583 closed 2 years ago

nimnp6583 commented 2 years ago

Thanks for developing this great tool. Its making our lives much easier while deploying adf code.

I'm facing trouble with "Delete objects not in source" option.

I have three pipelines (pipeline1, pipeline2, pipeline3) in target adf already I want to deploy only pipeline2(updated), delete pipeline3 and no changes to pipeline1.

note: pipeline3 has been deleted from source. So it has to be deleted from target as well.

So I have updated my rules.txt as below -pipeline.pipeline1 +pipeline.pipeline2

And I have selected "Delete objects not in source" option as I want to delete pipeline3 And also selected "Do not delete excluded objects" (in Advanced tab) as I don't want to delete excluded pipeline1 from deployment.

But pipeline3 is not deleted from target when I run the cicd pipeline. If i uncheck "Do not delete excluded objects" then pipeline3 from target adf is getting deleted.

When I check the log says

Removing excluded object: [Pipeline].[pipeline3] as publish option 'DoNotDeleteExcludedObjects' = false.

As per the rules.txt the excluded object is pipeline1 but not sure why it is saying excluded object as pipeline3 in the logs.

Is the option "Do not delete excluded objects" applicable for objects not in source (pipeline3 in this case) or the objects excluded from deployment as per rules.txt?

I'm attaching rules.txt and logs file with or without selecting "Do not delete excluded objects".

tasklog_6.log tasklog_6_Do_not_delete_excluded_objects.log rules (1).txt

NowinskiK commented 2 years ago

First of all, you can't use include and exclude rules together. See documentation: https://github.com/SQLPlayer/azure.datafactory.tools#publish-options

Bear in mind that Includes and Excludes lists are rules out each other. Objects would be excluded from deployment only if Includes list remains empty. When both lists are empty - all objects going to be published.

In your case, you don't have to use "Do not delete excluded objects" option because pipeline1 still exists in the source, so will not be deleted in the target even though it is excluded from the deployment.