Support for per-shard volumeClaimTemplate in clickhouse

jvilhuber commented 1 month ago

Name and Version

bitnami/clickhouse 6.2.4

What is the problem this feature will solve?

I have multiple existing PV's defined, which have labels to identify them. Say there's 2 PVs, with label "app: clickhouse-shard0" and "app: clickhouse-shard1".

Now I create a clickhouse deployment with replicaCount=1 and shards=2.

There's an outer range loop here: https://github.com/bitnami/charts/blob/main/bitnami/clickhouse/templates/statefulset.yaml#L7 which sets the shard index to $i:

{{- range $i, $e := until $shards }}

But I can't figure out if it's possible to use this variable in the render 'function call' in https://github.com/bitnami/charts/blob/main/bitnami/clickhouse/templates/statefulset.yaml#L450

        selector: {{- include "common.tplvalues.render" (dict "value" $.Values.persistence.selector "context" $) | nindent 10 }}

Basically, I want to be able to set the persistence.selector with a matchLabels section that would point to the PVs, but I can't feed in the shard-index. So for example I want to do something like:

"persistence":
  "enabled": true
  "selector":
    matchLabels:
      app: '{{ printf "clickhouse-shard%d" $shardIndex }}'

but of course $shardIndex isn't a variable.

It would be extremely handy to be able to access the shard index ($i) somehow in the volumeClaimTemplate creation.

What is the feature you are proposing to solve the problem?

Using the persistence.selectors to have shard-aware values.

What alternatives have you considered?

None come to mind.

monixca345 commented 1 month ago

That’s what I was looking for, what an info present here at this website, thank you admin! Login | Jetnet

andresbono commented 3 weeks ago

Hi @jvilhuber, to be honest I don't think it is possible to access to the outer range index $i...

jvilhuber commented 3 weeks ago

No idea either. I'm not really well versed in go templates, but I was hoping it could just be added into this dict here somehow:

( dict "value" $.Values.extraVolumeClaimTemplates "context" $)

In the meantime I've discovered a sort-of-workaround: An STS will only create the PVC(s) (from the volumeClaimTemplate) if one does not already exist (by name). So if I manage to get in there first (another story, that) and create the PVC(s) myself (and the PV) matching the exact names that the STS will use, then I can attach whatever I want of my choosing. Doing so without fail and race-conditions may be a different matter and depend on what you're using to deploy.

andresbono commented 3 weeks ago

Thank you for sharing your workaround, @jvilhuber.

github-actions[bot] commented 5 days ago

This Issue has been automatically marked as "stale" because it has not had recent activity (for 15 days). It will be closed if no further activity occurs. Thanks for the feedback.

github-actions[bot] commented 15 hours ago

Due to the lack of activity in the last 5 days since it was marked as "stale", we proceed to close this Issue. Do not hesitate to reopen it later if necessary.

bitnami / charts