Check whether job has any files before dispatching.

m-lab / etl-gardener

Gardener provides services for maintaining and reprocessing mlab data.

Apache License 2.0

13 stars 5 forks source link

Check whether job has any files before dispatching. #311

Closed gfr10598 closed 3 years ago

gfr10598 commented 3 years ago

This adds functionality to check whether a job has files in GCS before dispatching to ETL parser. This avoids launching a job that would otherwise result in BQ load failing.

This change is

coveralls commented 3 years ago

Coverage increased (+0.2%) to 59.608% when pulling 4dc257b3f8d1c27f9902f745d56390a3e0015a76 on u-sandbox-check-empty into 480c30dc4ef4a736da553316b53ed1603024996c on master.