Deploy fails when dependant LS does not exist yet

[NowinskiK]

When I try to deploy new Linked Services to empty Data Factory (all need to be created), the process fails for the first time with the following error message: ` Found 3 linked service(s) definitions.

Deploy linked service 'LS_ADLS'.
Deploy linked service 'LS_AzureKeyVault'.
Deploy linked service 'LS_SQL_Stackoverflow'.

[error]Error deploying 'LS_ADLS' linked service : {"error":{"code":"BadRequest","message":"The document creation or update failed because of invalid reference 'ls_azurekeyvault'.","target":"/subscriptions/0278080f-e1af-4ee8-98b3-881a286350aa/resourceGroups/rg-blog-uat/providers/Microsoft.DataFactory/factories/adf-blog-uat/linkedservices/LS_ADLS","details":null}}

`

I have 3 Linked Services, where 'LS_AzureKeyVault' should be deployed first, as the others use it to get secrets as property.
Changing a name (to 'AzureKeyVault') did not help - I thought that maybe it's a sorting problem. When I reviewed the code - I believe that all tasks are run in 5 parallel threads. This might be a serious issue when deploying dependant ADF objects.

Let me know if you couldn't reproduce the issue.

[liprec]

The deploy task doesn't parse the input and therefor it doesn't determine dependency; the major reason behind that is the task doesn't need to know the JSON schema and will keep working if the schema is changed regarding dependency e.g.

If you need dependency, you have to build it yourself. Several options are:

• Define order by prefixing the filenames, e.g. 01_
• Combine sort order and change the parallel tasks in the advanced section to force a strict order
• Create multiple steps to deploy the JSONs in the correct order

Currently I am thinking about to add multiple stages to the deploy task for easier maintenance.

-JP

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[NowinskiK]

Jan, due to this limitation, I created Custom Task for Azure DevOps Pipeline from scratch which is free of such limitations: DataFactoryTools
Forgive me for pointing here to another tool, but I believe it's worth mentioning because many people struggling with the same issue, i.e. dependencies. IMHO, renaming objects is not the correct solution people should go towards.

[liprec]

The current version (v2.3.3) contains support for the detection of pipeline/linked service dependencies in combination of parallel task execution. You can add detectDependancy: true to your pipeline definition to enable the detection.

[84salv84]

activating "Detect object dependancy" in release pipeline, the error persists