Nifikop pod is crashing when sample cluster yaml file is applied .

Vishwaja-optit commented 5 months ago

What steps will reproduce the bug?

Tried the solution provided in #376 ticket. But , still seeing the same error

Kubectl get po -n nifi-stage NAME READY STATUS RESTARTS AGE nifi-zookeeper-nifi-stage-0 1/1 Running 0 64s nifikop-stage-77b7499d9f-sxvnz 1/1 Running 0 2m58s

Applied the below yaml file:

apiVersion: nifi.konpyutaika.com/v1 kind: NifiCluster metadata: name: simplenifi spec: service: headlessEnabled: true labels: cluster-name: simplenifi zkAddress: "nifi-zookeeper-nifi-stage.nifi-stage:2181" zkPath: /simplenifi externalServices:

metadata: labels: cluster-name: driver-simplenifi name: driver-ip spec: portConfigs:
- internalListenerName: http port: 8080 type: ClusterIP clusterImage: "apache/nifi:1.24.0" initContainerImage: "bash:5.2.2" oneNifiNodePerNode: true readOnlyConfig: nifiProperties: overrideConfigs: | nifi.sensitive.props.key=thisIsABadSensitiveKeyPassword pod: labels: cluster-name: simplenifi nodeConfigGroups: default_group: imagePullPolicy: IfNotPresent isNode: true serviceAccountName: default storageConfigs:
  - mountPath: "/opt/nifi/nifi-current/logs" name: logs pvcSpec: accessModes:
    - ReadWriteOnce storageClassName: "gp3" resources: requests: storage: 10Gi resourcesRequirements: limits: cpu: "1" memory: 2Gi requests: cpu: "1" memory: 2Gi nodes:
id: 1 nodeConfigGroup: "default_group"
id: 2 nodeConfigGroup: "default_group" propagateLabels: true nifiClusterTaskSpec: retryDurationMinutes: 10 listenersConfig: internalListeners:
- containerPort: 8080 type: http name: http
- containerPort: 6007 type: cluster name: cluster
- containerPort: 10000 type: s2s name: s2s
- containerPort: 9090 type: prometheus name: prometheus
- containerPort: 6342 type: load-balance name: load-balance

After applying this yaml, No pod has come up and getting below error in nifikop pod

[Jan 29 2024 19:06:45 GMT+0530] nifikop-stage-77b7499d9f-sxvnz: {"level":"info","time":"2024-01-29T13:36:45.867Z","caller":"controller/controller.go:118","msg":"Observed a panic in reconciler: runtime error: invalid memory address or nil pointer dereference","controller":"nificluster","controllerGroup":"nifi.konpyutaika.com","controllerKind":"NifiCluster","nifiCluster":{"name":"simplenifi","namespace":"nifi-stage"},"namespace":"nifi-stage","name":"simplenifi","reconcileID":"a1c60f6f-7d8b-45ee-893b-784570276dfd"} [Jan 29 2024 19:06:45 GMT+0530] nifikop-stage-77b7499d9f-sxvnz: panic: runtime error: invalid memory address or nil pointer dereference [recovered] [Jan 29 2024 19:06:45 GMT+0530] nifikop-stage-77b7499d9f-sxvnz: panic: runtime error: invalid memory address or nil pointer dereference [Jan 29 2024 19:06:45 GMT+0530] nifikop-stage-77b7499d9f-sxvnz: [signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x10c3aa0] [Jan 29 2024 19:06:45 GMT+0530] nifikop-stage-77b7499d9f-sxvnz: goroutine 407 [running]: [Jan 29 2024 19:06:45 GMT+0530] nifikop-stage-77b7499d9f-sxvnz: sigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).Reconcile.func1() [Jan 29 2024 19:06:45 GMT+0530] nifikop-stage-77b7499d9f-sxvnz: /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.6/pkg/internal/controller/controller.go:119 +0x1a4 [Jan 29 2024 19:06:45 GMT+0530] nifikop-stage-77b7499d9f-sxvnz: panic({0x12adfe0?, 0x23a7090?}) [Jan 29 2024 19:06:45 GMT+0530] nifikop-stage-77b7499d9f-sxvnz: /usr/local/go/src/runtime/panic.go:914 +0x218 [Jan 29 2024 19:06:45 GMT+0530] nifikop-stage-77b7499d9f-sxvnz: github.com/konpyutaika/nifikop/pkg/resources/nifi.(Reconciler).Reconcile(0x400094ab40, {{0x17874c8, 0x40002dd920}, 0x0, 0x1, {0x0, 0x0}, {0x400054c270, 0x17}, {0x17736e0, ...}, ...}) [Jan 29 2024 19:06:45 GMT+0530] nifikop-stage-77b7499d9f-sxvnz: /workspace/pkg/resources/nifi/nifi.go:165 +0x1f60 [Jan 29 2024 19:06:45 GMT+0530] nifikop-stage-77b7499d9f-sxvnz: github.com/konpyutaika/nifikop/controllers.(NifiClusterReconciler).Reconcile(0x400007d2b0, {0x1786c00, 0x4000a42570}, {{{0x4000930070?, 0x30?}, {0x4000930060?, 0xffff71e70701?}}}) [Jan 29 2024 19:06:45 GMT+0530] nifikop-stage-77b7499d9f-sxvnz: /workspace/controllers/nificluster_controller.go:146 +0xd80 [Jan 29 2024 19:06:45 GMT+0530] nifikop-stage-77b7499d9f-sxvnz: sigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).Reconcile(0x1786c00?, {0x1786c00?, 0x4000a42570?}, {{{0x4000930070?, 0x11fb7e0?}, {0x4000930060?, 0x0?}}}) [Jan 29 2024 19:06:45 GMT+0530] nifikop-stage-77b7499d9f-sxvnz: /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.6/pkg/internal/controller/controller.go:122 +0x8c [Jan 29 2024 19:06:45 GMT+0530] nifikop-stage-77b7499d9f-sxvnz: sigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).reconcileHandler(0x40000c4320, {0x1786c38, 0x400003a690}, {0x1318520?, 0x4000a2cec0?}) [Jan 29 2024 19:06:45 GMT+0530] nifikop-stage-77b7499d9f-sxvnz: /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.6/pkg/internal/controller/controller.go:323 +0x294 [Jan 29 2024 19:06:45 GMT+0530] nifikop-stage-77b7499d9f-sxvnz: sigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).processNextWorkItem(0x40000c4320, {0x1786c38, 0x400003a690}) [Jan 29 2024 19:06:45 GMT+0530] nifikop-stage-77b7499d9f-sxvnz: /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.6/pkg/internal/controller/controller.go:274 +0x198 [Jan 29 2024 19:06:45 GMT+0530] nifikop-stage-77b7499d9f-sxvnz: sigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).Start.func2.2() [Jan 29 2024 19:06:45 GMT+0530] nifikop-stage-77b7499d9f-sxvnz: /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.6/pkg/internal/controller/controller.go:235 +0x74 [Jan 29 2024 19:06:45 GMT+0530] nifikop-stage-77b7499d9f-sxvnz: created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).Start.func2 in goroutine 102 [Jan 29 2024 19:06:45 GMT+0530] nifikop-stage-77b7499d9f-sxvnz: /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.6/pkg/internal/controller/controller.go:231 +0x43c [Jan 29 2024 19:06:46 GMT+0530] nifikop-stage-77b7499d9f-sxvnz: {"level":"info","time":"2024-01-29T13:36:46.640Z","caller":"workspace/main.go:78","msg":"WATCH_NAMESPACE ENV provided, will watch and manage resources in defined namespaces","namespaces":"nifi-stage"}

What is the expected behavior?

2 pods with default node group configuration should be in running state

What do you see instead?

[Jan 29 2024 19:06:45 GMT+0530] nifikop-stage-77b7499d9f-sxvnz: {"level":"info","time":"2024-01-29T13:36:45.867Z","caller":"controller/controller.go:118","msg":"Observed a panic in reconciler: runtime error: invalid memory address or nil pointer dereference","controller":"nificluster","controllerGroup":"nifi.konpyutaika.com","controllerKind":"NifiCluster","nifiCluster":{"name":"simplenifi","namespace":"nifi-stage"},"namespace":"nifi-stage","name":"simplenifi","reconcileID":"a1c60f6f-7d8b-45ee-893b-784570276dfd"} [Jan 29 2024 19:06:45 GMT+0530] nifikop-stage-77b7499d9f-sxvnz: panic: runtime error: invalid memory address or nil pointer dereference [recovered] [Jan 29 2024 19:06:45 GMT+0530] nifikop-stage-77b7499d9f-sxvnz: panic: runtime error: invalid memory address or nil pointer dereference [Jan 29 2024 19:06:45 GMT+0530] nifikop-stage-77b7499d9f-sxvnz: [signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x10c3aa0] [Jan 29 2024 19:06:45 GMT+0530] nifikop-stage-77b7499d9f-sxvnz: goroutine 407 [running]: [Jan 29 2024 19:06:45 GMT+0530] nifikop-stage-77b7499d9f-sxvnz: sigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).Reconcile.func1() [Jan 29 2024 19:06:45 GMT+0530] nifikop-stage-77b7499d9f-sxvnz: /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.6/pkg/internal/controller/controller.go:119 +0x1a4 [Jan 29 2024 19:06:45 GMT+0530] nifikop-stage-77b7499d9f-sxvnz: panic({0x12adfe0?, 0x23a7090?}) [Jan 29 2024 19:06:45 GMT+0530] nifikop-stage-77b7499d9f-sxvnz: /usr/local/go/src/runtime/panic.go:914 +0x218 [Jan 29 2024 19:06:45 GMT+0530] nifikop-stage-77b7499d9f-sxvnz: github.com/konpyutaika/nifikop/pkg/resources/nifi.(Reconciler).Reconcile(0x400094ab40, {{0x17874c8, 0x40002dd920}, 0x0, 0x1, {0x0, 0x0}, {0x400054c270, 0x17}, {0x17736e0, ...}, ...}) [Jan 29 2024 19:06:45 GMT+0530] nifikop-stage-77b7499d9f-sxvnz: /workspace/pkg/resources/nifi/nifi.go:165 +0x1f60 [Jan 29 2024 19:06:45 GMT+0530] nifikop-stage-77b7499d9f-sxvnz: github.com/konpyutaika/nifikop/controllers.(NifiClusterReconciler).Reconcile(0x400007d2b0, {0x1786c00, 0x4000a42570}, {{{0x4000930070?, 0x30?}, {0x4000930060?, 0xffff71e70701?}}}) [Jan 29 2024 19:06:45 GMT+0530] nifikop-stage-77b7499d9f-sxvnz: /workspace/controllers/nificluster_controller.go:146 +0xd80 [Jan 29 2024 19:06:45 GMT+0530] nifikop-stage-77b7499d9f-sxvnz: sigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).Reconcile(0x1786c00?, {0x1786c00?, 0x4000a42570?}, {{{0x4000930070?, 0x11fb7e0?}, {0x4000930060?, 0x0?}}}) [Jan 29 2024 19:06:45 GMT+0530] nifikop-stage-77b7499d9f-sxvnz: /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.6/pkg/internal/controller/controller.go:122 +0x8c [Jan 29 2024 19:06:45 GMT+0530] nifikop-stage-77b7499d9f-sxvnz: sigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).reconcileHandler(0x40000c4320, {0x1786c38, 0x400003a690}, {0x1318520?, 0x4000a2cec0?}) [Jan 29 2024 19:06:45 GMT+0530] nifikop-stage-77b7499d9f-sxvnz: /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.6/pkg/internal/controller/controller.go:323 +0x294 [Jan 29 2024 19:06:45 GMT+0530] nifikop-stage-77b7499d9f-sxvnz: sigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).processNextWorkItem(0x40000c4320, {0x1786c38, 0x400003a690}) [Jan 29 2024 19:06:45 GMT+0530] nifikop-stage-77b7499d9f-sxvnz: /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.6/pkg/internal/controller/controller.go:274 +0x198 [Jan 29 2024 19:06:45 GMT+0530] nifikop-stage-77b7499d9f-sxvnz: sigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).Start.func2.2() [Jan 29 2024 19:06:45 GMT+0530] nifikop-stage-77b7499d9f-sxvnz: /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.6/pkg/internal/controller/controller.go:235 +0x74 [Jan 29 2024 19:06:45 GMT+0530] nifikop-stage-77b7499d9f-sxvnz: created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).Start.func2 in goroutine 102 [Jan 29 2024 19:06:45 GMT+0530] nifikop-stage-77b7499d9f-sxvnz: /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.6/pkg/internal/controller/controller.go:231 +0x43c [Jan 29 2024 19:06:46 GMT+0530] nifikop-stage-77b7499d9f-sxvnz: {"level":"info","time":"2024-01-29T13:36:46.640Z","caller":"workspace/main.go:78","msg":"WATCH_NAMESPACE ENV provided, will watch and manage resources in defined namespaces","namespaces":"nifi-stage"}

Possible solution

No response

NiFiKop version

v1.6.0

Golang version

go1.19

Kubernetes version

1.27

NiFi version

1.16

Additional context

linked ticket #376

mh013370 commented 5 months ago

This appears to be the same scenario discussed in the slack: https://konpytika.slack.com/archives/C0362VBRM24/p1702911625342399

Please make sure your CRDs are updated to the v1.6.0 version. Primary indicator was:

Umar Hussain 1 month ago I think my crds are not updated. I cannot find reclaimPolicy in the cluster (edited)

Vishwaja-optit commented 5 months ago

I updated my crds and nifikop with 1.7 version. Now sample code is working. I tried with the helm chart of nifi-cluster in this repo. That is not working , it is just creating secret and nifi-cluster manifest But pods , pvc , service are not getting deployed.

https://github.com/konpyutaika/nifikop/tree/master/helm/nifi-cluster

Vishwaja-optit commented 4 months ago

Do we need to change anything in helm chart as it is not working. Nothing is coming up. No way to check logs or errors.

konpyutaika / nifikop