Infinite loop when deleting a linkedService and a connected dataset together

[PHPirates]

When we use this task to delete both a linkedService and a dataset which is (the only one) connected to that linkedService, the log shows the following:

2021-09-16T09:04:05.8865725Z ===================================================================================
2021-09-16T09:04:05.8869948Z STEP: Deleting objects not in source ...
2021-09-16T09:04:06.5479018Z Azure Data Factory (instance) loaded.
(...)
2021-09-16T09:04:15.2170716Z Removing object: [LinkedService].[MyLinkedService]
2021-09-16T09:04:15.7585012Z Removing object: [LinkedService].[MyLinkedService]
...
...
...
2021-09-16T09:09:21.6009482Z Removing object: [LinkedService].[MyLinkedService]
2021-09-16T09:09:22.1714808Z Removing object: [LinkedService].[MyLinkedService]
2021-09-16T09:09:25.4885314Z ##[error]The script failed due to call depth overflow.

After I delete the dataset manually, the deployment works and deletes the linked service correctly. Maybe this task should first delete datasets, and then linked services? Not sure why it runs into an infinite loop for five minutes.

[NowinskiK]

The app checks dependencies between objects before deleting any. Are you able to share the steps to reproduce the error? I mean: Deploy particular objects, then remove one of them and redeploy again with this issue. Without that, I won't be able to spend time investigating that as I believe it must be a specific edge case.

[PHPirates]

@NowinskiK We reproduced the error as follows:

• Add a linked service, a dataset referencing the linked service, and a pipeline containing an activity which references the dataset. These were all deployed together.
• Then we removed all of them, tried to deploy, and the issue occurred.

I then manually deleted these elements from the target data factory, to ensure that these were causing the issue (of course the error message mentioned the linked service, but still), after the manual deletion it deployed correctly.

I then reproduced the issue when trying to deploy the same code to a different target data factory, and that time I only deleted the pipeline and dataset manually first, before doing a successful deployment.

If you say that deleting a linked service should work and that this particular issue may be caused by some other property of the elements involved, I can try to create a minimal example and give you the json of the relevant files and steps to reproduce.

[NowinskiK]

Thanks, but can you show me that in PowerShell script? Please include demo files. I really would like to investigate all the issues, but don't have too much time to preparing the environment: files, script to repro, etc.

[NowinskiK]

I've just released a new version (1.7.*) - please check out whether the issue still exists.

[NowinskiK]

I'm closing this now as not heard any updates from @PHPirates. Feel free to reopen the ticket if this is still legitimate.

[PHPirates]

I'm trying it out just now, but I still have to figure out how to use to latest version, and actually to find out which version we're using at all (as we are using it via the DevOps Task in a Release pipeline).

The log says

### azure.datafactory.tools                                       Version 0.91.000 ###

so that doesn't seem to be up to date, if it's the right version number.

[NowinskiK]

This is the correct version. Another release has just out a few minutes ago.

[NowinskiK]

The precise version information is there: https://marketplace.visualstudio.com/items?itemName=SQLPlayer.DataFactoryTools#:~:text=Test%20Connection%20Task-,Related%20modules,-These%20tasks%20include

[PHPirates]

@NowinskiK Aha got it, I was confusing the two version numbers. We were originally still using the ancient version 0.90.792 and now we are using 1.8.937, which works a lot better and fixes this particular issue as well. Thanks for the support!

2021-10-01T11:35:19.3175377Z ==============================================================================
2021-10-01T11:35:19.3175834Z Task         : Publish Azure Data Factory
2021-10-01T11:35:19.3176192Z Description  : Deploys entire ADF (v2) from JSON files to ADF instance
2021-10-01T11:35:19.3176567Z Version      : 0.90.792
2021-10-01T11:35:19.3176834Z Author       : Kamil Nowinski (SQLPlayer)
2021-10-01T11:35:19.3177285Z Help         : [More Information](https://github.com/SQLPlayer/azure.datafactory.devops)
2021-10-01T11:35:19.3177713Z ==============================================================================

2021-10-01T12:12:49.4670419Z ==============================================================================
2021-10-01T12:12:49.4670937Z Task         : Publish Azure Data Factory
2021-10-01T12:12:49.4671252Z Description  : Deploys entire ADF (v2) from JSON files to ADF instance (adftools)
2021-10-01T12:12:49.4671537Z Version      : 1.8.937
2021-10-01T12:12:49.4671753Z Author       : Kamil Nowinski (SQLPlayer)
2021-10-01T12:12:49.4672110Z Help         : [More Information](https://github.com/SQLPlayer/azure.datafactory.devops)
2021-10-01T12:12:49.4672485Z ==============================================================================

[NowinskiK]

Happy to help. Bear in mind that the current version is 1.9.942: