external_table.md and 5 images: Manual steps to run
Code changes
-deploy_infrastructure.sh: Add extra storage env to deploy_synapse_artifacts.sh
-deploy_synapse_artifacts.sh: Add "upload SQL Script function" to upload sql script to synapse workspace
-04_explore_interim.ipynb: Notebook to explorer data when external db. View and user needed to setup correctly before running this notebook
-create_db_user.sql: Manual steps to update value and run manually in synapse after deployment =>ToDo for Future improvement: Run this sql command while deployment after setting sqladmin. but need to install sqlcommand in dev container
create_external_table_template.sql: Need to run this script after there is data file in data/interim folder to be able to create external view. Need to run pipeline at least once before running this script=> ToDo for Future improvement: To be able run while deployment, either create folder in advance in deployment or need to run pipeline first then run this sql script
Purpose
Purpose is to be able to create a view(schema less) to be able to view data and query view on SQL Serverless from notebook
Author pre-publish checklist
[ ] Added test to prove my fix is effective or new feature works
[ ] No PII in logs
[ x] Made corresponding changes to the documentation
Type of PR
Purpose
Purpose is to be able to create a view(schema less) to be able to view data and query view on SQL Serverless from notebook
Author pre-publish checklist
Validation steps
Please refer to the document