Extend sample synapse serverless - Githubissues

Azure-Samples / modern-data-warehouse-dataops

DataOps for the Modern Data Warehouse on Microsoft Azure. https://aka.ms/mdw-dataops.

MIT License

570 stars 451 forks source link

Extend sample synapse serverless #611

Closed ydaponte closed 1 year ago

ydaponte commented 1 year ago

Extending the synapse serverless sample to include the following:

json configuration file that dynamically drives the ingestion behavior and security access applied to the datalake hierarchy
data retention demonstration with an assumption of (2 years) as a stand alone synapse pipeline that cleans the data lake accordingly

DoD

Change the current sample to include:

json configuration file

Smart and dynamic ingestion inline with the configuration file definitions

Complete and tested automation of the ACLs application through a synapse notebook

Complete and tested automation of the Data Retention scenario through a synapse notebook

Updated documentation