Added Mysql Database to Snowflake data set workflow - Githubissues

openedx-unsupported / edx-analytics-pipeline

GNU Affero General Public License v3.0

91 stars 116 forks source link

Added Mysql Database to Snowflake data set workflow #754

Closed HassanJaveed84 closed 5 years ago

HassanJaveed84 commented 5 years ago

Differentiate between Snowflake DB name and MySQL DB name. Improve logging upon failed Snowflake query. Insert a date into each stage name, so different stages can point to different dates of the same DB export.

Relies on an ExternalURL task.

Analytics Pipeline Pull Request

Make sure that the following steps are done before merging:

[ ] If you have a migration please contact data engineering team before merging.
[ ] Before merging run full acceptance tests suite and provide URL for the acceptance tests run.
[ ] A member of data engineering team has approved the pull request.

HassanJaveed84 commented 5 years ago

This is ready for a final review, along with https://github.com/edx/jenkins-job-dsl/pull/792