konpyutaika / nifikop

The NiFiKop NiFi Kubernetes operator makes it easy to run Apache NiFi on Kubernetes. Apache NiFI is a free, open-source solution that support powerful and scalable directed graphs of data routing, transformation, and system mediation logic.
https://konpyutaika.github.io/nifikop/
Apache License 2.0
131 stars 45 forks source link

Refactor NifiCluster state management CRD Configuration #358

Open mh013370 opened 8 months ago

mh013370 commented 8 months ago

Is your feature request related to a problem?

No

Describe the solution you'd like to see

With NiFi 2.0.0, the zookeeper configuration becomes optional with the addition of kubernetes-based cluster state management. Update the NifiCluster CRD to:

  1. Make zookeeper configuration optional
  2. Consolidate cluster state management configuration in the CRD

The following changes should be made to v2alpha1/NifiCluster:

apiVersion: nifi.konpyutaika.com/v2alpha1
kind: NifiCluster
metadata:
  name: nifi-cluster
spec:
  stateManagement:
    kubernetes:
      identifier: my-cluster
    zookeeper:
      zkPath: /my-cluster
      zkAddress: zookeeper.zookeeper:2181

One of kubernetes or zookeeper should be configured and this should be enforced by the operator.

Update all documentation to reflect these changes.

Describe alternatives you've considered

No response

Additional context

This issue requires #359 to be completed first.

juldrixx commented 8 months ago

A field should exist to chose which one (zk or k8s) you want to use, no?

mh013370 commented 8 months ago

I should give two examples. I believe it can be an either/or. So you'd configure either:

apiVersion: nifi.konpyutaika.com/v2alpha1
kind: NifiCluster
metadata:
  name: nifi-cluster
spec:
  stateManagement:
    zookeeper:
      zkPath: /my-cluster
      zkAddress: zookeeper.zookeeper:2181

OR

apiVersion: nifi.konpyutaika.com/v2alpha1
kind: NifiCluster
metadata:
  name: nifi-cluster
spec:
  stateManagement:
    kubernetes:
      identifier: my-cluster

And the existence of one or the other is how nifikop will know which to configure.