Invalid arguments were passed to DbtSeedOperator

vymaker commented 1 year ago

I am using dbt + MWAA v2.2. Following your instructions https://github.com/gocardless/airflow-dbt#templating-and-parsing-environments-variables, I set up the following dag:

dbt_run = DbtSeedOperator(
   task_id="dbt_seed",
   dbt_bin='/usr/local/airflow/.local/bin/dbt',
   profiles_dir='/usr/local/airflow/dags/{DBT_FOLDER}/',
   dir='/usr/local/airflow/dags/{DBT_FOLDER}/',
   env={
    'DBT_ENV_SECRET_DATABASE': '<DATABASE>',
    'DBT_ENV_SECRET_PASSWORD': '<PASSWORD>',
    'DBT_ENV_SECRET_SCHEMA': '<SCHEMA>',
    'USER_NAME': '<USER_NAME>',
  }
)

And for some reason I get such an error which I do not know how to debug, any help you could provide? """ airflow.exceptions.AirflowException: Invalid arguments were passed to DbtSeedOperator (task_id: dbt_seed). """ My env is fine since I have another dag (without airflow-dbt) that works. Assume something to do with the args. Moreover, here I do not find support for ENV arg: https://pypi.org/project/airflow-dbt/#description.

So I assume it has not been released yet? If no, when is it planned? And is there any workaround?

Thanks in advance!

emily-flambe commented 1 year ago

@vymaker I faced a similar frustration as you regarding use of the env arg - as you can see there is some code that's been merged into the main branch (though actually has a bug preventing it from working as intended), but the latest release was about 18 months ago!

My team went ahead and forked the repo so we could fix the bug and make a new PyPi release. At some point we will likely spin up our own repo - this was kind of a desperation measure - but anyway the repo is here, and the pypi package is here. Maybe give those a spin and see if you can get it working for your own needs?

tarush-bitstrapped commented 1 year ago

Even i am facing same issue with dbtrunoperator

gocardless / airflow-dbt

Invalid arguments were passed to DbtSeedOperator #77