ClickHouse / dbt-clickhouse

The Clickhouse plugin for dbt (data build tool)
Apache License 2.0
245 stars 100 forks source link

Support skip indexes #358

Open stephen-up opened 2 weeks ago

stephen-up commented 2 weeks ago

Hi, it would be great if dbt-ch supported skip indexes, much like it does projections.

Eg

{{ config(  
  engine='MergeTree()',
  order_by='id,
  materialized='table',
  indexes=[{
    'name': 'minmax_received_ts',
    'column': 'received_ts',
    'type': 'minmax',
    'granularity': '2'
  }],
) }}

That would run statements like

ALTER TABLE {{ this }} ADD INDEX IF NOT EXISTS {{ name }} {{ column }} TYPE {{ type }} GRANULARITY {{ granularity }};
ALTER TABLE {{ this }} MATERIALIZE INDEX {{ name }};

Currently, I have a work around, and define them in post_hooks but it times out sometimes.

Thanks

Is your feature request related to a problem? Please describe. We have trouble with long run times, poor performance and stability.

Describe the solution you'd like It would be great if dbt supported skip indexes, like it does projections.