zilliztech / milvus-operator

The Kubernetes Operator of Milvus.
https://milvus.io
Apache License 2.0
33 stars 20 forks source link

External Minio connectivity issue #102

Open jacobpaul1893 opened 3 months ago

jacobpaul1893 commented 3 months ago

I am trying to use an external minio for Milvus in standalone mode, but I am getting an access denied error. I followed instructions here to create the config - https://milvus.io/docs/object_storage_operator.md Here is the manifest I used

apiVersion: milvus.io/v1beta1
kind: Milvus
metadata:
  name: milvus-standalone
  namespace: milvus
spec:
  components:
    image: milvusdb/milvus:v2.4.0-rc.1
    standalone:
      serviceType: NodePort
    volumes:
      - name: tls
        secret:
          secretName: milvus-tls
      - name: cacert
        configMap:
          name: cacert.pem
    volumeMounts:
      - name: tls
        mountPath: /var/lib/milvus/cert
      - name : cacert
        mountPath: /var/lib/milvus/cacert
  dependencies:
    etcd:
      inCluster:
        values:
          image:
            pullPolicy: IfNotPresent
            repository: milvusdb/etcd
            tag: 3.5.5-r4
          persistence:
            enabled: true
            accessMode: ReadWriteOnce
            size: 10Gi
            storageClass: storage-class
          replicaCount: 1
          extraEnvVars:
            - name: ETCD_MAX_REQUESTS_BYTES
              value: "1572864"
    storage:
      external: true
      type: "MinIO"
      endpoint: "sidekick-minio:9000"
      secretRef: "my-minio-secret"
  config:
    common:
      security:
        authorizationEnabled: true
        tlsMode: 2
    tls:
      serverPemPath: /var/lib/milvus/cert/tls.crt
      serverKeyPath: /var/lib/milvus/cert/tls.key
      caPemPath: /var/lib/milvus/cacert/cacert.pem
    minio:
      bucketName: milvus-data
      rootPath: milvus-test
      useSSL: false

Error from Milvus

Status:
  Conditions:
    Last Transition Time:  2024-03-29T22:28:16Z
    Message:               Etcd endpoints is healthy
    Reason:                EtcdReady
    Status:                True
    Type:                  EtcdReady
    Last Transition Time:  2024-03-29T22:28:16Z
    Message:               checkMinIO with backoff failed: Access Denied.
    Reason:                ClientError
    Status:                False
    Type:                  StorageReady
    Last Transition Time:  2024-03-29T22:28:16Z
    Message:               MsgStream is ready
    Reason:                MsgStreamReady
    Status:                True
    Type:                  MsgStreamReady
    Last Transition Time:  2024-03-29T22:28:16Z
    Message:               [standalone] not ready, detail: component[standalone]: deployment not created
    Reason:                MilvusComponentNotHealthy
    Status:                False
    Type:                  MilvusReady
    Last Transition Time:  2024-03-29T22:28:16Z
    Message:               Milvus components[standalone] are updating
    Reason:                MilvusComponentsUpdating
    Status:                False
    Type:                  MilvusUpdated
  Ingress:
    Load Balancer:
  Observed Generation:  1
  Replicas:
  Status:  Pending 

I had used the same set of credentials to access the same Minio bucket using aws cli.

jacobpaul1893 commented 3 months ago

I was able to connect to external Minio by using storage.type as "S3" instead of "MinIO"