Altinity / clickhouse-operator

Altinity Kubernetes Operator for ClickHouse creates, configures and manages ClickHouse® clusters running on Kubernetes
https://altinity.com
Apache License 2.0
1.94k stars 466 forks source link

AWS EBS IOPS and Throughput - Possbiity of adding annotations on volumeclaim template #1552

Closed meektechie closed 2 weeks ago

meektechie commented 2 weeks ago

Hi Team,

We are trying to increase IOPS & Throughput of the EBS volumes which is attached to the clickhouse pods to improve the performance. AWS suggesting couple of things

  1. Fix it through storage-class - This is a global settings (not applicable)
  2. Adding annotations to the PV annotations: "ebs.csi.aws.com/volumeType": "gp3" "ebs.csi.aws.com/iops": "5000" "ebs.csi.aws.com/throughput": "250"

https://aws.amazon.com/blogs/storage/simplifying-amazon-ebs-volume-migration-and-modification-using-the-ebs-csi-driver/

I went through the clickhouseinstallation CR but i dont see any such. Please let me know if we have any such facility.

Slach commented 2 weeks ago

did you try

apiVersion: clickhouse.altinity.com/v1
kind: ClickHouseInstallation
spec:
 defaults:
   templates:
     dataVolumeClaimTemplate: default
 templates:
   volumeClaimTemplates:
    - metadata:
        annotations:
          "ebs.csi.aws.com/volumeType": "gp3"
          "ebs.csi.aws.com/iops": "5000"
          "ebs.csi.aws.com/throughput": "250"
      name: default
      reclaimPolicy: Retain
      spec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 500Gi

it should works for modern EBS CSI Driver

if this is not works for your EKS installation then first and common solution is define storage class with CSI driver-specific annotations and use it

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  annotations:
  name: gp3
allowVolumeExpansion: true
parameters:
  encrypted: "true"
  fsType: ext4
  iops: "7000"
  throughput: "1000"
  type: gp3
provisioner: ebs.csi.aws.com
reclaimPolicy: Delete
volumeBindingMode: WaitForFirstConsumer

---
apiVersion: clickhouse.altinity.com/v1
kind: ClickHouseInstallation
metadata:
  name: example
spec:
  defaults:
    templates:
      dataVolumeClaimTemplate: my-specific-storage-class
  templates:
    volumeClaimTemplates:
    - name: my-specific-storage-class
      spec:
        storageClassName: my-specific-storage-class  

second, try to use https://github.com/Altinity/ebs-params-controller and use

 volumeClaimTemplates:
    - metadata:
        annotations:
          spec.epc.altinity.com/iops: "5000"
          spec.epc.altinity.com/throughput: "1000"
      name: default
      reclaimPolicy: Retain
      spec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 500Gi
meektechie commented 2 weeks ago

It works. Thanks for the quick support