Open malyq opened 5 years ago
Commit ef544b06b117534661ded19f09fdce197f917486
Oozie workflows have been reworked. Library now includes Setup and Execution sections. Setup is run once from development environment to save sqoop jobs in sqoop metastore. Execution workflows must be migrated into EMR.
Current iteration still missing biforce-execute-imports.xml workflow and biforce-execute.properties file. This file should have subworkflow actions to call hive-execute-import.xml and warehouse-execute-import.xml.
Automate the entire process from Caliber to Redshift, beginning with transferring the data to S3 and connecting to EMR.