search

apache / airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
https://airflow.apache.org/
Apache License 2.0
36.84k stars 14.25k forks source link

Feature Request - Sqoop hook and operators #197

Closed syvineckruyk closed 4 years ago

syvineckruyk commented 9 years ago

to be defined.

cgelman commented 9 years ago

I want to +1 this.

chenghaibo commented 9 years ago

I want to +1 this

rakeshsingh commented 8 years ago

I want to +1 this

notifytovishal commented 8 years ago

+1 Can this be done using https://pypi.python.org/pypi/sqoopy/ ? Can this be defined !!

syvineckruyk commented 8 years ago

We currently have a functioning SqoopOperator that we use internally. Currently the feature set is based on our need and may be considered rather limited. I would be interested to hear what the community is looking in terms of features from a SqoopOperator.

Currently our Sqoop Operator has the benefit of using the connections from Airflow ... so no need to to have your connection strings and credentials in your code or command parameter. It takes tool (import,export etc...), args, kwargs and templated option file as parameters.

hyperborea commented 8 years ago

@syvineckruyk any chance to get a sneak peak at your current implementation?

syvineckruyk commented 8 years ago

@hyperborea sure thing ... ill get a pr going and update here.

mafc commented 8 years ago

Any updates on this?

mistercrunch commented 8 years ago

https://github.com/apache/incubator-airflow/pull/1489