When using dbt-expectations with Clickhouse, it can't convert the timezone when tests require a datetime column. It seems that the convert_timezone function of dbt-date is not set up / specified for Clickhouse, so it uses the default command which is not compatible with Clickhouse.
Steps to reproduce
Set up dbt with Clickhouse
Add catalogica/dbt-expecations with dbt deps
Add a test that uses a datetime column, for example:
` tables:
name: test_table
columns:
name: date_col
tests:
dbt_expectations.expect_row_values_to_have_recent_data:
datepart: day
interval: 1`
Following error will appear:
Database Error in test dbt_expectations_source_expect_row_values_to_have_recent_data_dev_test_table_date_col__day__1 (models/gold/schema.yml) HTTPDriver for https://mc4yb4j21e.europe-west4.gcp.clickhouse.cloud:8443 returned response code 404) Code: 46. DB::Exception: Unknown function convert_timezone. Maybe you meant: ['serverTimezone','serverTimeZone']: While processing CAST(date_col, 'timestamp') <= convert_timezone('UTC', 'UTC', CAST(now(), 'DateTime')). (UNKNOWN_FUNCTION) (version 24.6.1.4609 (official build))
Expected behaviour
Test will run and process the timestamp column correctly
Code examples, such as models or profile settings
schema.yml
`version: 2
sources:
name: dev
tables:
name: test_table
columns:
name: date_col
tests:
dbt_expectations.expect_row_values_to_have_recent_data:
datepart: day
interval: 1`
dbt and/or ClickHouse server logs
10:09:13 Running with dbt=1.8.7
10:09:13 Registered adapter: clickhouse=1.8.4
10:09:13 Found 1 model, 1 test, 1 source, 725 macros
10:09:13
10:09:14 Concurrency: 1 threads (target='dev')
10:09:14
10:09:14 1 of 1 START test dbt_expectations_source_expect_row_values_to_have_recent_data_dev_test_table_date_colday1 [RUN]
10:09:14 1 of 1 ERROR dbt_expectations_source_expect_row_values_to_have_recent_data_dev_test_table_date_colday1 [ERROR in 0.06s]
10:09:14
10:09:14 Finished running 1 test in 0 hours 0 minutes and 1.29 seconds (1.29s).
10:09:14
10:09:14 Completed with 1 error and 0 warnings:
10:09:14
10:09:14 Database Error in test dbt_expectations_source_expect_row_values_to_have_recent_data_dev_test_table_date_colday1 (models/gold/schema.yml)
HTTPDriver for https://mc4yb4j21e.europe-west4.gcp.clickhouse.cloud:8443 returned response code 404)
Code: 46. DB::Exception: Unknown function convert_timezone. Maybe you meant: ['serverTimezone','serverTimeZone']: While processing CAST(date_col, 'timestamp') <= convert_timezone('UTC', 'UTC', CAST(now(), 'DateTime')). (UNKNOWN_FUNCTION) (version 24.6.1.4609 (official build))
Describe the bug
When using dbt-expectations with Clickhouse, it can't convert the timezone when tests require a datetime column. It seems that the convert_timezone function of dbt-date is not set up / specified for Clickhouse, so it uses the default command which is not compatible with Clickhouse.
Steps to reproduce
Database Error in test dbt_expectations_source_expect_row_values_to_have_recent_data_dev_test_table_date_col__day__1 (models/gold/schema.yml) HTTPDriver for https://mc4yb4j21e.europe-west4.gcp.clickhouse.cloud:8443 returned response code 404) Code: 46. DB::Exception: Unknown function convert_timezone. Maybe you meant: ['serverTimezone','serverTimeZone']: While processing CAST(date_col, 'timestamp') <= convert_timezone('UTC', 'UTC', CAST(now(), 'DateTime')). (UNKNOWN_FUNCTION) (version 24.6.1.4609 (official build))
Expected behaviour
Test will run and process the timestamp column correctly
Code examples, such as models or profile settings
schema.yml `version: 2
sources:
dbt and/or ClickHouse server logs
10:09:13 Running with dbt=1.8.7 10:09:13 Registered adapter: clickhouse=1.8.4 10:09:13 Found 1 model, 1 test, 1 source, 725 macros 10:09:13
10:09:14 Concurrency: 1 threads (target='dev') 10:09:14
10:09:14 1 of 1 START test dbt_expectations_source_expect_row_values_to_have_recent_data_dev_test_table_date_colday1 [RUN] 10:09:14 1 of 1 ERROR dbt_expectations_source_expect_row_values_to_have_recent_data_dev_test_table_date_colday1 [ERROR in 0.06s] 10:09:14
10:09:14 Finished running 1 test in 0 hours 0 minutes and 1.29 seconds (1.29s). 10:09:14
10:09:14 Completed with 1 error and 0 warnings: 10:09:14
10:09:14 Database Error in test dbt_expectations_source_expect_row_values_to_have_recent_data_dev_test_table_date_colday1 (models/gold/schema.yml) HTTPDriver for https://mc4yb4j21e.europe-west4.gcp.clickhouse.cloud:8443 returned response code 404) Code: 46. DB::Exception: Unknown function convert_timezone. Maybe you meant: ['serverTimezone','serverTimeZone']: While processing CAST(date_col, 'timestamp') <= convert_timezone('UTC', 'UTC', CAST(now(), 'DateTime')). (UNKNOWN_FUNCTION) (version 24.6.1.4609 (official build))
Configuration
Environment
ClickHouse server
CREATE TABLE
statements for tables involved: