MicrosoftLearning / dp-203-azure-data-engineer

Exercise files for Microsoft Data Engineer curriculum
https://microsoftlearning.github.io/dp-203-azure-data-engineer/
MIT License
359 stars 410 forks source link

Unable to provision databricks resources with setup.ps1 #109

Closed atetz closed 1 month ago

atetz commented 1 month ago

Module: Implement a Data Analytics Solution with Azure Databricks

Lab/Demo: 24,25,26

Task: Provisioning resources

Step: 05

Description of issue I have been unable to provision the resources using the provided script: 'MicrosoftLearning/mslearn-databricks/setup.ps1'

The error in the shell: Trying eastus... 0 of 10 cores in use. Creating msl-1wkyfh8 resource group ... Creating databricks-1wkyfh8 Azure Databricks workspace in msl-1wkyfh8 resource group... New-AzDatabricksWorkspace_CreateExpanded: /usr/local/share/powershell/Modules/Az.Databricks/1.8.0/Databricks.Autorest/custom/New-AzDatabricksWorkspace.ps1:421 Line | 421 | Az.Databricks.internal\New-AzDatabricksWorkspace @PSBound … | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | RESOURCE_DOES_NOT_EXIST: The Workspace Creator with App ID: b677c290-cf4b-4a8e-a60e-91ba650a4abe Could | Not Be Found Granting permissions on the databricks-1wkyfh8 resource... (you can ignore any warnings!)

The workspace is created but the overview page of the databricks service shows the following error: The workspace 'databricks-1wkyfh8' is in a failed state and cannot be launched. Please review error details in the activity log tab and retry your operation.

The activity log shows the following error at 'Initialize Databricks Workspace': Operation name Initialize Databricks workspace Time stamp Wed Jul 24 2024 09:03:22 GMT+0200 (Central European Summer Time) Event initiated by Databricks Resource Provider Error code BadRequest Message {"error":{"code":"ResourceDoesNotExist","message":"RESOURCE_DOES_NOT_EXIST: The Workspace Creator with App ID: b677c290-cf4b-4a8e-a60e-91ba650a4abe Could Not Be Found"}}

I have also tried to run the script with a preferred location as argument './setup.ps1 westeurope'. But my argument get's skipped with the following information: Trying westeurope... NotAvailableForSubscription westeurope has insufficient capacity.

Which is strange because I have no other EDSv4 cores in use in westeurope and my quota has a maximum of 10.

Repro steps:

Step 1 to 5 in: https://microsoftlearning.github.io/mslearn-databricks/Instructions/Exercises/04-Azure-Databricks-SQL.html

GraemeMalcolm commented 1 month ago

I'm not able to repro this in my personal Azure subscription. Are you seeing this in a hosted environment? It's possible there's a policy constraint that's preventing creation of the workspace assets (which have changed since the lab was originally written) - @LODS-41-319 , is this maybe something in the cloud slice?

TheJamesHerring commented 1 month ago

@atetz - has this problem been resolved per @GraemeMalcolm's question? I will close tomorrow as I was able to build in personal azure subscription with no issue. If it does persist, I will have tester check in Europe. James H.

TheJamesHerring commented 1 month ago

Closing as appeared to be a local/regional or potentially a BYOL issue, which is not what these files are for. JH