Open iordaniordanov opened 2 years ago
Hello, any info here ?
Hello, yes this is the expected behaviour, we are forced to because if all the init cluster nodes are down, it could lead in the situation where the new joining node decides to be the reference, and in this case all information would be erased from the other nodes once they rejoin ... So we have an init script specific for new node, and once the node has explicitly joined the cluster, we need to restart the pod with a "non-joining" script : https://github.com/Orange-OpenSource/nifikop/blob/master/pkg/resources/nifi/pod.go#L392
Okey, thanks for the clarification :)
I'm sure you thought it trough, but just a suggestion - maybe before scaling up you can check if the cluster reports that it is healthy and if it is not abort the scale operation, because otherwise if someone wants to add lets say 50 nodes because of a spike in usage in this case he needs to wait for multiple hours before all nodes successfully join the cluster ...
Type of question
About general context and help around nifikop
Question
What did you do? Increased number of nodes in the nificluster CR from 3 to 6
What did you expect to see? 3 new nodes to be simultaneously created and joined in the cluster
What did you see instead? Under which circumstances? 3 new nodes were simultaneously created, they join the cluster, but after that they are one by one re-created and only after that the cluster is fully functional, which leads to a linear increase in the amount of time which is needed to scale the cluster up. If adding one node takes 5 min adding 2 nodes takes ~10 min and so on. Is this the expected behavior or it is an issue with our configuration/environment ?
Environment
nifikop version:
v0.6.0
Kubernetes version information:
Server Version: version.Info{Major:"1", Minor:"17+", GitVersion:"v1.17.17-eks-087e67", GitCommit:"087e67e479962798594218dc6d99923f410c145e", GitTreeState:"clean", BuildDate:"2021-07-31T01:39:55Z", GoVersion:"go1.13.15", Compiler:"gc", Platform:"linux/amd64"}
Kubernetes cluster kind: EKS
NiFi version: 1.12.1
Additional context Nifi cluster config