AtlasOfLivingAustralia / pipelines-airflow

About Airflow DAGs and supporting files for running pipelines on Apache Airflow with Elastic Map Reduce.
Other
0 stars 1 forks source link

image sync download all datasets into the cluster #11

Open patkyn opened 1 month ago

patkyn commented 1 month ago

Image sync downloaded all datasets causing a long running emr cluster instead of the relevant datasets

image