search

ClickHouse / dbt-clickhouse

The Clickhouse plugin for dbt (data build tool)
Apache License 2.0
245 stars 100 forks source link

Insert query that is OOM'ing in Clickhouse is returning OK to dbt #360

Open barberscott opened 2 weeks ago

barberscott commented 2 weeks ago

Describe the bug

I have an insert query that is consisently out of memory-ing. In the Clickhouse logs I can see an exception 241 being thrown on the query.

In dbt, I see:

13:36:37  dbt_clickhouse adapter: SQL status: OK in 136.20 seconds

In query_log I see:

Code: 241. DB::Exception: Memory limit (total) exceeded: would use 146.96 GiB (attempt to allocate chunk of 68720781687 bytes), maximum: 100.80 GiB. OvercommitTracker decision: Query was selected to stop by OvercommitTracker.: While executing FillingRightJoinSide. (MEMORY_LIMIT_EXCEEDED) (version 24.5.1.22957 (official build))

and the problem is that dbt then proceeds despite reaching the OOM -- I would expect dbt to fail here. This seems to have started to happen on/around September 4 or so.

barberscott commented 2 weeks ago

Near as I can tell, this only seems to impact OOMs on inserts. OOMs on select are appropriately reporting state.