Adds the deployment of a Synapse workspace and a Synapse Spark Pool as a baseline to port the parking sensor sample to synapse
Adds the deployment of a Log Analytics workspace with saved kusto query to analyze notebooks logs later
Automates update of the synapse workspace to contain needed whl packages located on /synapse/libs folder
Configures spark pool to include whl packages
Configures spark pool to point to requirements.txt that is located on synapse/config
Does this introduce a breaking change? If yes, details on what can break
Not completely a breaking change. On the main.bicep file I'm referencing to a storage_v2.bicep module instead of the previous storage.bicep version. Reason is we are deploying a second storage account to be the default sa for the synapse workspace.
Author pre-publish checklist
[ X] No PII in logs
[ X] Made corresponding changes to the documentation
Validation steps
Deployed the e2e sample with the changes, full sample deployment was successful.
Access to Synapse workspace in all environments and check spark pool configuration settings and package settings for requirements.txt
Check if Log Analytics workspace (Id and Key) are correctly referenced on the spark pool configuration for each environment
On the Azure Portal, navigate to the deployed synapse workspace/spark pool/Packages and check if Requirement files is filled
with requirements.txt
On the Azure Portal, navigate to the deployed synapse workspace/spark pool/Packages and check if Workspace packages are filled with the following file: ddo_transform-localdev-py2.py3-none-any.whl.
Type of PR
Purpose
Does this introduce a breaking change? If yes, details on what can break
Not completely a breaking change. On the main.bicep file I'm referencing to a storage_v2.bicep module instead of the previous storage.bicep version. Reason is we are deploying a second storage account to be the default sa for the synapse workspace.
Author pre-publish checklist
Validation steps
Issues Closed or Referenced