search

google / orchestra

Advertising Data Lakes and Workflow Automation
Apache License 2.0
48 stars 27 forks source link

Migrate GoogleCloudStorageToFTPOperator to core operators #14

Closed mik-laj closed 4 years ago

mik-laj commented 4 years ago

Hello

My team created a new operators for Airflow that copy files from and to GCS and FTPS. https://github.com/apache/airflow/pull/6393 https://github.com/apache/airflow/pull/6366

These operators can be replaced by an operator that has been implemented by your team. https://github.com/google/orchestra/blob/Orchestra-2.0/orchestra/google/cloud/operators/gcp_gcs_operator.py I think that it makes no sense to duplicate operators and it is better to use the operator that will be provided with the new versions of Airflow/Cloud Composer.

Best regards,

CC: @TobKed

KingsleyKelly commented 4 years ago

Hi mik-laj!

Thanks so much for your hard work on this operator. Our goal is to remove any operators that are not needed from this project in future versions including many GMP operators.

We can't provide any dates at the moment but we will be doing ongoing releases to update our code and use airflow operators where we can. Expect to see this change in a future release and we can cc you when it takes place.

Cheers!