dbt-labs / dbt-utils

Utility functions for dbt projects.
https://hub.getdbt.com/dbt-labs/dbt_utils/latest/
Apache License 2.0
1.34k stars 491 forks source link

Union Relations compilation doesn't include all columns #953

Open moseleyi opened 1 week ago

moseleyi commented 1 week ago

From what I read in the documentation, even when relations differ, the macro will result in a list of unique columns from all relations and fill things with NULL when necessary. However, that's not what's happening in my case.

Steps to reproduce

model_a.sql

SELECT
  2 AS test,
  3 AS test2

model_b.sql

SELECT
  1 AS test

Expected results

( 
  SELECT
    CAST('''`project.schema.model_a`''' AS STRING) AS _dbt_source_relation,
    CAST(test AS INT64) AS test,
    CAST(test2 AS INT64) AS test2
  FROM model_a
) UNION ALL
(
  SELECT
    CAST('''`project.schema.model_b`''' AS STRING) AS _dbt_source_relation,
    CAST(test AS INT64) AS test,
    CAST(NULL AS INT64) AS test2
  FROM model_b
)

Actual results

test2 column is not even present in the compiled code

( 
  SELECT
    CAST('''`project.schema.mnodel_a`''' AS STRING) AS _dbt_source_relation,
    CAST(test AS INT64) AS test
  FROM model_a
) UNION ALL
(
  SELECT
    CAST('''`project.schema.mnodel_b`''' AS STRING) AS _dbt_source_relation,
    CAST(test AS INT64) AS test
  FROM model_b
)

System information

The contents of your packages.yml file:

packages:
  - package: dbt-labs/dbt_utils
    version: 1.3.0

Which database are you using dbt with?

The output of dbt --version:

Core: 1.8.4
BigQuery: 1.8.2