search

MicrosoftDocs / azure-docs

Open source documentation of Microsoft Azure
https://docs.microsoft.com/azure
Creative Commons Attribution 4.0 International
10.26k stars 21.44k forks source link

Need clarity for databricks bundle deploy and bundle destroy #117266

Closed xhead closed 11 months ago

xhead commented 11 months ago

When repo assets are deployed to a databricks workspace, it is not clear if the "databricks bundle deploy" will remove files from the target workspace that aren't in the source repo. For example, if a repo contained a notebook named "test1.py" and had been deployed, but then "test1.py" was removed from the repo and a new notebook "test2.py" was created, what is the content of the target workspace after? I believe it will contain both "test1.py" and "test2.py".

Secondly, the description of "databricks bundle destroy" does not indicate that it would remove all files from the workspace - only that it will remove all the artifacts referenced by the bundle. So when the "test1.py" file has been removed from the repo, and the "databricks bundle destroy" is run, will it only remove "test2.py" (which has not yet been deployed)?

I am trying to determine how to ensure that the shared workspace contains only the files that are in the repo - that whatever I do in a release pipeline, I will only have the latest assets in the workspace that are in the repo, and none of the old files that were previously in the repo. The semantics of "databricks bundle deploy" (in particular the term "deploy") would indicate to me that it should do a clean up of assets in the target workspace as part of the deployment. But if that is not the case, then if I did a "databricks bundle destroy" prior to the "databricks bundle deploy", would that adequately clean up the target workspace? Or do I need to do something with "databricks fs rm" to delete all the files in the target workspace folder?

Document Details

⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

Naveenommi-MSFT commented 11 months ago

@xhead Thanks for your feedback! We will investigate and update as appropriate.

AjayBathini-MSFT commented 11 months ago

@xhead Thank you for your feedback! I'd recommend working closer with our support team via an [Azure support request] (https://docs.microsoft.com/en-us/azure/azure-portal/supportability/how-to-create-azure-support-request). Or you can leverage our Q&A forum by posting your issue there so our community, and MVPs can further assist you in troubleshooting this issue or finding potential workarounds. [Teams Q&A forum] (https://docs.microsoft.com/en-us/answers/topics/46488/office-teams-windows-itpro.html) for technical questions about the configuration and administration of Microsoft Teams on Windows. [Microsoft Teams Community forum] (https://answers.microsoft.com/en-us/msteams/forum?sort=LastReplyDate&dir=Desc&tab=All&status=all&mod=&modAge=&advFil=&postedAfter=&postedBefore=&threadType=All&isFilterExpanded=false&page=1) Thank you for your time and patience throughout this issue.