Altinity / clickhouse-operator

Altinity Kubernetes Operator for ClickHouse creates, configures and manages ClickHouse® clusters running on Kubernetes
https://altinity.com
Apache License 2.0
1.92k stars 461 forks source link

increase the max_suspicious_broken_parts_bytes not working #1505

Closed darioneto closed 2 months ago

darioneto commented 2 months ago

One of my pods is failing to start due to error DB::Exception: Suspiciously big size (8 parts, 1.37 GiB in total) of all broken parts to remove while maximum allowed broken parts size is 1.00 GiB.

I have already updated the configuration to the following:

apiVersion: "clickhouse.altinity.com/v1"
kind: "ClickHouseInstallation"
metadata:
  name: "ck3"
  namespace: ck
spec:
  defaults:
    templates: 
      dataVolumeClaimTemplate: default
      podTemplate: clickhouse:20.7-with-backup

  configuration:
    settings:
      merge_tree:
        max_suspicious_broken_parts_bytes: 4294967296
      max_suspicious_broken_parts_bytes: 4294967296
kubectl exec -n ck chi-ck3-replicated-0-0-0 -- sh -c "grep -r 'max_suspicious_broken_parts_bytes' /etc/clickhouse-server/"
Defaulted container "clickhouse-pod" out of: clickhouse-pod, clickhouse-backup
/etc/clickhouse-server/config.d/..2024_09_06_14_43_34.962645124/chop-generated-settings.xml:    <max_suspicious_broken_parts_bytes>4294967296</max_suspicious_broken_parts_bytes>

However, the current behavior still limits it to 1 GiB, preventing the attachment of certain tables. Could you please let me know how I can ensure that ClickHouse respects the new setting?

chengjoey commented 2 months ago

try:

spec:
  configuration:
    settings:
      merge_tree/max_suspicious_broken_parts_bytes: 4294967296
chi-addon-clickhouse-clickhouse-0-0-0:/# grep -r 'max_suspicious_broken_parts_bytes' /etc/clickhouse-server/ -C 20
/etc/clickhouse-server/config.d/chop-generated-settings.xml-<yandex>
/etc/clickhouse-server/config.d/chop-generated-settings.xml-    <merge_tree>
/etc/clickhouse-server/config.d/chop-generated-settings.xml:        <max_suspicious_broken_parts_bytes>4294967296</max_suspicious_broken_parts_bytes>
/etc/clickhouse-server/config.d/chop-generated-settings.xml-    </merge_tree>
/etc/clickhouse-server/config.d/chop-generated-settings.xml-</yandex>

reference docs, it should be working

darioneto commented 2 months ago

thanks for your prompt response, indeed it worked :-)