Recently running some stuff on fileflow running airflow 1.7.1.3 I was getting deprecation warnings for the following:
/Users/llorenz/Envs/fileflow/lib/python2.7/site-packages/airflow/models.py:1719: PendingDeprecationWarning: Invalid arguments were passed to DivePythonOperator. Support for passing such arguments will be dropped in Airflow 2.0. Invalid arguments were:
*args: ()
**kwargs: {'data_dependencies': {'something': 'write_a_file'}}
category=PendingDeprecationWarning
According to the dev email list this is anticipation of only supporting explicit args:
Hi Laura,
The error is raised if an unused argument is passed to BaseOperator --
basically if there is anything in either args or kwargs. The original issue
was that in a number of cases arguments were misspelled or misused by
Operator subclasses and instead of raising an error, they were just passed
up the inheritance chain and finally (silently) absorbed by BaseOperator,
so there was no warning.
I think a workaround should be straightforward -- when you call
super().init for the BaseOperator, just pass arguments explicitly
rather than with args/kwargs, or (alternatively), pop arguments out of
kwargs when you use them ahead of calling that init.
Our case might be a little harder because we're subclassing PythonOperator not BaseOperator directly, but we can start at this point to see how we want to workaround for Airflow 2.0. That being said, Airflow 2.0 probably isn't coming out for another... year?... so we have some time.
Recently running some stuff on fileflow running airflow 1.7.1.3 I was getting deprecation warnings for the following:
According to the dev email list this is anticipation of only supporting explicit args:
Our case might be a little harder because we're subclassing
PythonOperator
notBaseOperator
directly, but we can start at this point to see how we want to workaround for Airflow 2.0. That being said, Airflow 2.0 probably isn't coming out for another... year?... so we have some time.