search

MicrosoftLearning / dp-203-azure-data-engineer

Exercise files for Microsoft Data Engineer curriculum
https://microsoftlearning.github.io/dp-203-azure-data-engineer/
MIT License
402 stars 456 forks source link

Cluser version is invalid. #130

Closed sumahmud closed 2 weeks ago

sumahmud commented 3 weeks ago

Module: 07

Lab/Demo: 07

Task: Provision an Azure Synapse Analytics workspace

Step: 05

Description of issue The cluster creation has been failed as it throws the error that spark compute version 3.1 is invalid. I have checked the setup script and the version is set to 3.1. Once I created the cluster manually from the studio with a lower version, it worked fine.

image

tylerfarmer1 commented 2 weeks ago

I had the same issue. I believe the fix is in the file setup.json, line 56. It currently says:

  "sparkVersion": "3.1"

I just changed it to 3.4 instead of 3.1 and the script worked fine.

I also showed my students how to create the Spark Cluster manually, which also makes the lab work fine.

GraemeMalcolm commented 2 weeks ago

This should be resolved by a PR I just merged