search

nats-io / nats-streaming-operator

NATS Streaming Operator
Apache License 2.0
174 stars 44 forks source link

Proper PV support - ReadWriteOnce #23

Open andreimc opened 5 years ago

andreimc commented 5 years ago

Hey,

I tried your PV support with EFS RWX but it wasn't that good so I have a proposed implementation for PV support inside the operator.

eg

# Default values for nats-streaming-cluster
nats:
  replicas: 3
  clusterId: "nats-cluster"

stan:
  replicas: 3
  storage:
    size: 10Gi
    keepOnDelete: true
  clusterId: "nats-streaming-cluster"

where it actually creates the pvs with the pods themselves or mounts them if they exist? The only thing I am not sure about what happens when you delete the operator you could leave the volumes hanging around if you delete your cluster?

What do you guys think?

wallyqs commented 5 years ago

sounds interesting so a PV would be created for each one of the pods? Maybe could use owner references to delete the volumes when pods go away?

andreimc commented 5 years ago

@wallyqs yeah that's what I am thinking, I will get time later this week to have a crack at this.

Lazyshot commented 5 years ago

This sounds absolutely necessary as we need to run NATS Streaming in an HA manner.

C0en commented 5 years ago

What he said ^

Maybe we can have a volume claimtemplate like in statefulsets, so each node gets a seperate volumeclaim with a number and then use a storage class to provision the volumes. Proposed like this:

apiVersion: "streaming.nats.io/v1alpha1" kind: "NatsStreamingCluster" metadata: name: "example-stan-pv" spec: size: 3 natsSvc: "example-nats" config: storeDir: "/pv/stan" template: spec: containers:

wallyqs commented 5 years ago

The volumeClaimTemplates idea to make it provision PV dynamically per requested instance I think sounds good

pvanderlinden commented 5 years ago

Currently you can only have a cluster of servers without persistence or just a single server with persistence? The volumeClaimTemplates would be a good idea to fix this indeed, as it will be consistent with Statefulsets

rajugupta15 commented 5 years ago

Is any update on this?

snayakie commented 4 years ago

Hi, I just want to confirm that the nats-streaming-operator doesn't support statefulset stylevolumeClaimTemplates in the NatsStreamingCluster crd , does it?

I just tried the following, and it doesn't seem to work....so just want to confirm that its not supported.

---
apiVersion: "streaming.nats.io/v1alpha1"
kind: "NatsStreamingCluster"
metadata:
  name: "stan-pvc-template"
spec:
  size: 3
  natsSvc: "example-nats"

  config:
    storeDir: "/pv/stan"

  volumeClaimTemplates:
    - metadata:
        name: streaming-pvc
      spec:
        storageClassName: "local-storage"
        accessModes:
          - ReadWriteOnce
        volumeMode: Filesystem
        resources:
          requests:
            storage: 128Mi

  # Define mounts in the Pod Spec
  template:
    spec:
      containers:
        - name: nats-streaming
          volumeMounts:
          - mountPath: /pv/stan
            name: streaming-pvc