Filter settings based on Engine

ClickHouse / dbt-clickhouse

The Clickhouse plugin for dbt (data build tool)

Apache License 2.0

253 stars 113 forks source link

Filter settings based on Engine #367

Open shachibista opened 1 month ago

shachibista commented 1 month ago

This PR introduces a feature that ignores settings that are not compatible with the Engine. More details in this issue: https://github.com/ClickHouse/dbt-clickhouse/issues/366

Checklist

[x] A human-readable description of the changes was provided to include in CHANGELOG

CLAassistant commented 1 month ago

All committers have signed the CLA.

BentsiLeviav commented 3 weeks ago

Hi @shachibista,

Thank you for your contribution!

I'm hesitant to move forward with this change as it introduces ongoing maintenance requirements. With this setup, every time ClickHouse adds new settings, we’ll need to update the ENGINE_SETTINGS dictionary. Similarly, any new engine additions would also require corresponding updates.

I think the liability is greater than the benefit.

BentsiLeviav commented 2 weeks ago

@genzgd suggested changing it from a white list to a black list (remove settings we identified as not relevant for this engine), and in case we face a new engine, pass the settings as is.

@shachibista would you mind changing the logic to support this?