Add Spark job S3 upload method

ccao-data / service-spark-iasworld

Service for extracting tables from the CCAO system-of-record and uploading them to the Data Department's data warehouse

GNU Affero General Public License v3.0

0 stars 0 forks source link

Closed dfsnow closed 2 months ago

dfsnow commented 2 months ago

This PR adds an upload() method to SparkJob that takes Hive-partitioned table extraction results and uploads them to S3.