Synapse Serverless pools have some limitations when compared with SQL dedicated pools. The aim of this sample is to demonstrate how to work with Serverless pools and Delta files, how to automate the view creation of top of a Data Lake folder structure holding the Delta tables and finally how to apply CLS (Column-Level Security) on top of the Lake without the need of using third-party solutions.
Proposed Solution
Deploy a Synapse workspace with the Built-in Serverless pool and artifacts that will implement the following:
Import an open-source parquet format dataset
Convert parquet to delta format
Dynamically create partitioned views on top of the delta lake tables folder structure using Synapse pipelines
Apply CLS to the columns in the Delta table through the use of AD Groups using the Control plane in Azure and Synapse and Data Plane in Synapse.
Problem Statement
Proposed Solution
Deploy a Synapse workspace with the Built-in Serverless pool and artifacts that will implement the following:
Alternative Proposals
Additional Context
Next Steps