Azure-Samples / modern-data-warehouse-dataops

DataOps for the Modern Data Warehouse on Microsoft Azure. https://aka.ms/mdw-dataops.

MIT License

570 stars 451 forks source link

Fix data source and data context for great expectations #486

Closed sudivate closed 2 years ago

sudivate commented 2 years ago

Type of PR

Code changes

Purpose

Transformation step had incorrect data source and data context name for great expectations checkpoint.

Does this introduce a breaking change? If yes, details on what can break

Author pre-publish checklist

[ ] Added test to prove my fix is effective or new feature works
[ x] No PII in logs
[ ] Made corresponding changes to the documentation

YueSheng6 commented 2 years ago

@devlace Shall we merge this PR? It fixes the error when running Databricks Notebook03 on great expectations. "runError": "DatasourceError: Cannot initialize datasource transformed_data_source, error: The given datasource could not be retrieved from the DataContext; please confirm that your configuration is accurate."

Either making changes in data context like this PR or changing the names in BatchRequest will work.