Closed ilias1111 closed 2 months ago
delta
iceberg
dbt_project.yml
date_diff
date_add
timestamp_functions.sql
bigint
get_value_by_target_type
spark
get_string_agg
get_field
QUALIFY
CAST(x AS data_type)
ROW_NUMBER()
Data Lakes Epic
Description
delta
withiceberg
as an alternative for Spark targets.dbt_project.yml
to include the file format configuration for models based on the target type.date_diff
anddate_add
functions intimestamp_functions.sql
, casting results asbigint
to prevent overflow.get_value_by_target_type
macro to includespark
as a valid option.get_string_agg
andget_field
macros to ensure compatibility.QUALIFY
clause with PostgreSQL-compatible logic to ensure consistent behavior across different target types.CAST(x AS data_type)
for Spark compatibility.ROW_NUMBER()
in tests to account for Spark's non-deterministic behavior.What type of PR is this? (check all applicable)
Related Tickets & Documents
Data Lakes Epic
Checklist
Added tests?
Added to documentation?
[optional] What gif best describes this PR or how it makes you feel?