dbt-labs / dbt-core

dbt enables data analysts and engineers to transform their data using the same practices that software engineers use to build applications.
https://getdbt.com
Apache License 2.0
9.25k stars 1.53k forks source link

[Bug] Can't pass `args` for `run-operation` in programmatic invocation as kwarg #10355

Open ben-schreiber opened 5 days ago

ben-schreiber commented 5 days ago

Is this a new bug in dbt-core?

Current Behavior

Passing the --args parameter as a kwarg to the dbtRunner.invoke method throws a TypeError. This forces the parameter to only be pass as:

dbtRunner().invoke(['run-operation', '--args', '...'])

This looks to be caused by a naming conflict with the invoke method here

Expected Behavior

To also be able to pass it as a kwarg

dbtRunner().invoke(['run-operation'], args=...)

Steps To Reproduce

from dbt.cli.main import dbtRunner

runner = dbtRunner()
runner.invoke(['run-operation', 'some_macro'], args={'arg1': 'foo', 'arg2': 'goo'})

Relevant log output

TypeError
...
TypeError: dbtRunner.invoke() got multiple values for argument 'args'

Environment

- OS: MacOS Sonoma 14.4
- Python: 3.10.12
- dbt: 1.8.3

Which database adapter are you using with dbt?

snowflake

Additional Context

No response

ben-schreiber commented 2 days ago

Here are two suggestions which came to mind:

  1. The args parameter could be renamed. Something like,

    def invoke(self, invocation_args: List[str], **kwargs) -> dbtRunnerResult: ...

    Although, this would be a breaking change and would need to be handled accordingly.

  2. To add a dedicated name for this use-case (e.g. macro_args or something like that) which then would be translated back to args

    def invoke(self, args: List[str], **kwargs) -> dbtRunnerResult:
    if 'macro_args' in kwargs:
        kwargs['args'] = kwargs.pop('macro_args')

    This too would have to be appropriately documented since it breaks from the kwargs convention of this method.