bug: volume resize does not work

josedev-union commented 3 weeks ago

Description

I provisioned the cluster with the following spec and it works.

DorisCluster spec

```yaml feSpec: replicas: 2 image: selectdb/doris.fe-ubuntu:2.1.2 service: type: LoadBalancer persistentVolumes: - mountPath: /opt/apache-doris/fe/doris-meta name: fetest persistentVolumeClaimSpec: accessModes: - ReadWriteOnce resources: requests: storage: 20Gi - mountPath: /opt/apache-doris/fe/log name: felog persistentVolumeClaimSpec: accessModes: - ReadWriteOnce resources: requests: storage: 10Gi beSpec: replicas: 3 image: selectdb/doris.be-ubuntu:2.1.2 service: type: LoadBalancer persistentVolumes: - mountPath: /opt/apache-doris/be/storage name: betest persistentVolumeClaimSpec: accessModes: - ReadWriteOnce resources: requests: storage: 40Gi - mountPath: /opt/apache-doris/be/log name: belog persistentVolumeClaimSpec: accessModes: - ReadWriteOnce resources: requests: storage: 20Gi ```

I changed the storage size for all volumes and the DorisCluster CR spec has been updated. But it is not applied to underlying resources like statefulset and PVCs. I feel like there is no reconcile logic for that.

There is no error logs in doris operator. Here is the logs.

doris-operator log

```sh I0607 09:52:22.652461 1 doriscluster_controller.go:91] DorisClusterReconciler reconcile the update crd name test-c1 namespace doris I0607 09:52:22.652602 1 client.go:35] CreateOrUpdateService service Name, Ports, Selector, ServiceType, Labels have not change namespace doris name test-c1-fe-internal I0607 09:52:22.652670 1 client.go:35] CreateOrUpdateService service Name, Ports, Selector, ServiceType, Labels have not change namespace doris name test-c1-fe-service I0607 09:52:22.653179 1 statefulset.go:96] the statefulset name test-c1-fe new hash value 1386567562 old have value 1386567562 I0607 09:52:22.653196 1 client.go:54] ApplyStatefulSet Sync exist statefulset name=test-c1-fe, namespace=doris, equals to new statefulset. I0607 09:52:22.653247 1 client.go:35] CreateOrUpdateService service Name, Ports, Selector, ServiceType, Labels have not change namespace doris name test-c1-be-internal I0607 09:52:22.653365 1 client.go:35] CreateOrUpdateService service Name, Ports, Selector, ServiceType, Labels have not change namespace doris name test-c1-be-service I0607 09:52:22.653812 1 statefulset.go:96] the statefulset name test-c1-be new hash value 3597274972 old have value 3597274972 I0607 09:52:22.653826 1 client.go:54] ApplyStatefulSet Sync exist statefulset name=test-c1-be, namespace=doris, equals to new statefulset. I0607 09:52:22.653837 1 controller.go:201] Doris cluster is not have cn I0607 09:52:22.653845 1 controller.go:201] Doris cluster is not have cn ```

Also DorisCluster status is available.

NAME     FESTATUS    BESTATUS    CNSTATUS   BROKERSTATUS
test-c1   available   available

Expectation

THe volume size change in DorisCluster spec should be applied.

How to reproduce

Create a cluster with any size of volume
Increase that volume size after the cluster is up

intelligentfu commented 3 weeks ago

Hi, doris operator uses statefulset to manage pods, the statefulset is not allowed to update PVC size, now. doris operator is adding a webhook for hook updating PVC size. for some extreme situations, PVC can be created by Operator When the provisioner field is set to 'Operator', as follows.

apiVersion: doris.selectdb.com/v1
kind: DorisCluster
metadata:
  name: doriscluster-sample-storageclass1
spec:
  feSpec:
    replicas: 3
    image: selectdb/doris.fe-ubuntu:2.1.1
    limits:
      cpu: 8
      memory: 16Gi
    requests:
      cpu: 8
      memory: 16Gi
    persistentVolumes:
    - mountPath: /opt/apache-doris/fe/doris-meta
      name: fetest
      provisioner: Operator
      persistentVolumeClaimSpec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 10Gi
    - mountPath: /opt/apache-doris/fe/log
      name: felog
      provisioner: Operator
      persistentVolumeClaimSpec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 10Gi
  beSpec:
    replicas: 3
    image: selectdb/doris.be-ubuntu:2.1.1
    limits:
      cpu: 8
      memory: 16Gi
    requests:
      cpu: 8
      memory: 16Gi
    persistentVolumes:
    - mountPath: /opt/apache-doris/be/storage
      name: betest
      provisioner: Operator
      persistentVolumeClaimSpec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 10Gi
    - mountPath: /opt/apache-doris/be/log
      name: belong
      provisioner: Operator
      persistentVolumeClaimSpec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 10Gi

This function is in beta and not described in docs. when using Operator mode, The management mode cannot updated anymore.

wdxxl commented 3 weeks ago

mark as same issues. using kubectl edit pv & pvc for increase storage size are not work.

using version: doris.be-ubuntu:2.1.3 & doris.k8s-operator:1.5.2

selectdb / doris-operator