Azure-Samples / modern-data-warehouse-dataops

DataOps for Microsoft Data Platform technologies. https://aka.ms/dataops-repo

MIT License

588 stars 459 forks source link

Add Parking Sensors Synapse Sample #448

Closed devlace closed 2 years ago

devlace commented 3 years ago

Type of PR

Documentation changes
Code changes
Test changes
CI-CD changes

Purpose

This completes milestone 1 of porting the existing Parking Sensor solution from Databricks into Azure Synapse.

User Journeys:

[X] Provision - Deploy sample to an Azure subscription
[X] ETL - Trigger pipeline to see the data flow to PBI.
[X] ETL dev - Modify a pipeline or notebook and commit it
[X] ETL dev - Modify Python code, trigger a push package
[X] ETL dev - Promote through Staging & Prod
[X] ETL dev - View integration tests to ensure everything is good
[X] Observability - Looking at logs to see how long execution took
[X] Data exploration - How can a Data Scientist use serverless to explore

References #406

Does this introduce a breaking change? If yes, details on what can break

Author pre-publish checklist

[X] Added test to prove my fix is effective or new feature works
[X] No PII in logs
[X] Made corresponding changes to the documentation

Issues Closed or Referenced

Closes #407
References #444, #439, #438, #436, #434, #432, #426, #424, #418, #417, #415, #414, #403, #401, #399, #398, #397, #396, #395, #394, #391, #390, #389