search

Azure-Player / azure.datafactory.tools

Tools for deploying Data Factory (v2) in Microsoft Azure
https://azureplayer.net/adftools
MIT License
207 stars 69 forks source link

Do not read or handle the issue with objects that are excluded from deletion process #341

Open MartaUch opened 1 year ago

MartaUch commented 1 year ago

Hello,

I would like to ask if it would be possible to not fully take into account objects which where excluded during deletion of not existing objects in the source. In general I see this feature works correctly, so for example if I don't want to remove linkedService then indeed it won't be deleted. Problem occurs when LinkedService for example is not created along with Microsoft recommendation (problem with privateKeyContent which is parametrized and not using Azure Key Vault - currently we can not avoid this approach and it can't be easily fixed; similar issue found: https://github.com/Azure-Player/azure.datafactory.tools/issues/85). As I understand your code reads all the objects (even excluded once) at the beginning and later decide whether object should be deleted or not.

Could that be possible to handle that kind of issue in the code? For example to catch the error and check first if it's even a type of the object that should be deleted. That would be really helpful.

Kind Regards Marta

MartaUch commented 10 months ago

Hi @NowinskiK :) Could that be possible to solve this problem in the upcoming months? I know it's not technically a bug, but from my perspective it's a little bit missleading that I'm excluding the object even from deletion process and it's still somehow taken into account - what leads to error than I'm expecting and that's why I'm excluding it. Kind regards Marta Uchimiak

NowinskiK commented 10 months ago

I'm not planning to work on this any soon. This is an edge-case and likely will make more harm in the code over benefits. If you have time and want to convince me I'm wrong - prepare changes and create PR. Generally, each ADF should have at least once Linked Service to AKV. If you say it's not possible (?) to do that - it sounds like "security is not important for an organisation", which is already bad start. In this case, the justification of the problem also becomes irrelevant.

MartaUch commented 10 months ago

Yes you are right, It’s a good point. I will be analyzing if we can implement such a cases correctly to not run into that types of errors anymore.

I understand also your point of view in this case, so If I have free time and problem on our side won’t be possible to fix, I will think about possible solution in your code and share that with you :) Thank you!