search

Orange-OpenSource / nifikop

The NiFiKop NiFi Kubernetes operator makes it easy to run Apache NiFi on Kubernetes. Apache NiFI is a free, open-source solution that support powerful and scalable directed graphs of data routing, transformation, and system mediation logic.
https://orange-opensource.github.io/nifikop/
Apache License 2.0
128 stars 34 forks source link

Controller NifiCluster in endless loop #146

Closed Arsou33 closed 2 years ago

Arsou33 commented 2 years ago

Bug Report

What did you do? Declaring a simple Nifi cluster with two nodes

What did you expect to see? Cluster is starting and NifiKop is stable

What did you see instead? Under which circumstances? Nifi cluster is correctly started but nifiklop NifiCluster controller flood logs log.txt

Environment

Additional context Cluster definition


apiVersion: nifi.orange.com/v1alpha1
kind: NifiCluster
metadata:
  name: nifi
spec:
  service:
    headlessEnabled: true
  zkAddress: "zookeeper:2181"
  zkPath: "/nifi"
  clusterImage: "apache/nifi:1.12.1"
  oneNifiNodePerNode: false
  nodeConfigGroups:
    default_group:
      isNode: true
      storageConfigs:
        - mountPath: "/opt/nifi/nifi-current/logs"
          name: logs
          pvcSpec:
            accessModes:
              - ReadWriteOnce
            storageClassName: "csi-cinder-high-speed"
            resources:
              requests:
                storage: 10Gi
      serviceAccountName: "default"
      resourcesRequirements:
        limits:
          cpu: "0.5"
          memory: 2Gi
        requests:
          cpu: "0.5"
          memory: 2Gi
  nodes:
    - id: 1
      nodeConfigGroup: "default_group"
    - id: 2
      nodeConfigGroup: "default_group"
  propagateLabels: true
  nifiClusterTaskSpec:
    retryDurationMinutes: 10
  listenersConfig:
    internalListeners:
      - type: "http"
        name: "http"
        containerPort: 8080
      - type: "cluster"
        name: "cluster"
        containerPort: 6007
      - type: "s2s"
        name: "s2s"
        containerPort: 10000
      - type: "prometheus"
        name: "prometheus"
        containerPort: 9090
  externalServices:
    - name: "clusterip"
      spec:
        type: ClusterIP
        portConfigs:
          - port: 8080
            internalListenerName: "http"
mh013370 commented 2 years ago

Are those logs not just the regular reconciliation loop?

Arsou33 commented 2 years ago

Could be. But it is 4 to 5 times a second generating about 35Kb/s of logs. Without debug level it is 10Kb/s of logs, leading to more that 1Gb per day.

mh013370 commented 2 years ago

That does seem a little chatty!

erdrix commented 2 years ago

Yes I'm agree on this point ... We kept the logic of the logging from the koperator, but it is really too verbose ... We start to refactor using the events and adding a way to configure the interval between reconcile loops : https://github.com/Orange-OpenSource/nifikop/pull/124, but it is far from good.

I would be happy if you have any ideas on how we could improve it :)