search

dbt-labs / spark-utils

Utility functions for dbt projects running on Spark
https://hub.getdbt.com/fishtown-analytics/spark_utils/latest/
Apache License 2.0
31 stars 15 forks source link

Better exception if unreal date/ts #9

Closed jtcohen6 closed 3 years ago

jtcohen6 commented 3 years ago

resolves #6 cc @emilieschario

Use assert_true() to help Spark give us a more helpful error message, if the user has supplied a date/timestamp that is not real (returns null when cast to date or timestamp).

models/bad_date_spine.sql

{{ dbt_utils.date_spine(
    datepart="day",
    start_date="'2020-02-30'",
    end_date="current_date"
) }}

Old error:

Compilation Error in model bad_date_spine (models/bad_date_spine.sql)
  '<=' not supported between instances of 'NoneType' and 'int'

  > in macro generate_series (macros/sql/generate_series.sql)
  > called by macro date_spine (macros/datetime/date_spine.sql)
  > called by macro get_powers_of_two (macros/sql/generate_series.sql)
  > called by model bad_dateadd (models/bad_date_spine.sql)

New error:

Runtime Error in model bad_date_spine (models/bad_date_spine.sql)
  Database Error
    java.lang.RuntimeException: 'isnotnull(cast(2020-02-30 as date))' is not true!