search

dbt-labs / dbt-external-tables

dbt macros to stage external sources
https://hub.getdbt.com/dbt-labs/dbt_external_tables/latest/
Apache License 2.0
286 stars 115 forks source link

Running dbt_external_tables under Airflow causing error with incorrect YAML format #283

Open lukasz-wiesek opened 2 months ago

lukasz-wiesek commented 2 months ago

Hello, I am running dbt with dbt_external_tables package under Composer (v2.4.3). When I am passing arguments to my class to build DBT command like: dbt run-operation stage_external_sources --args 'select: external_sources.abc_full' --target dev --profile abc_seg this command gives me an error: [2024-04-11, 12:06:53 UTC] {pod_manager.py:382} INFO - 12:06:53 The YAML provided in the --args argument is not valid. [2024-04-11, 12:06:53 UTC] {pod_manager.py:382} INFO - Usage: dbt run-operation [OPTIONS] MACRO [2024-04-11, 12:06:53 UTC] {pod_manager.py:382} INFO - Try 'dbt run-operation -h' for help. [2024-04-11, 12:06:53 UTC] {pod_manager.py:382} INFO - [2024-04-11, 12:06:53 UTC] {pod_manager.py:382} INFO - Error: Invalid value for '--args': String ''select:' is not valid YAML

I was trying to use curly braces like: --args '{select: external_sources.abc_full}' but it is not resolving an issue. When I am running this command without "--args" just pure macro stage_external_sources it is working perfectly even under Composer. How I can resolve issue with this?

dataders commented 2 months ago

@lukasz-wiesek this is strange for sure! Shot in the dark: try double quoting (") the string?