Open Sreenivas-Ratakonda opened 2 years ago
Hello @Sreenivas-Ratakonda !
I'm using headlessEnabled: true
. But when I scale the clusters for 2 and more, I can't access with user that is configured, it is returning the message "Purposed state does not match the stored state. Unable to continue login process."
I've configured ingress with nginx.ingress.kubernetes.io
but not work either.
@Sreenivas-Ratakonda . I've solved my problem.
In my case, I need to configure keycloack. It was missing to configure the option Authorization Enabled = true
Then, in your case, you need to configure headlessEnabled: true
, and in ingress inserted the configuration:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
nginx.ingress.kubernetes.io/affinity: "cookie"
nginx.ingress.kubernetes.io/session-cookie-expires: "172800"
nginx.ingress.kubernetes.io/session-cookie-max-age: "172800"
nginx.ingress.kubernetes.io/affinity-mode: persistent
@wandersonpereira
I have set the headless:true
and configured the alb load balencer instead of nignx with the help of ELB Ingress controller Now I am to Access the UI of nificluster I am not facing any invalid token issues either.
But I am still not able to create the nifiuser and nifigroups using the CRD mentioned in the docs. to be specific the problem with user creation is the users are not getting synchronized with the cluster
Please look at the example below:
For successful user creation: this user got created post OIDC auth:
I have created another user using the CRD:
apiVersion: nifi.orange.com/v1alpha1
kind: NifiUser
metadata:
name: sslnifi.bpeadmin
spec:
identity: bpeadmin@brillio.com
clusterRef:
name: sslnifi
namespace: nifi
createCert: true
includeJKS: true
secretName: bpeadmin_secrets
output:
Am I missing something Over here ...?
One more thing did you try to setup Site to Site with other nifi Instance.
@Sreenivas-Ratakonda .
Do you use anything to auth in your UI, exemple Keycloak, Azure, GCP or you use only users in Nifi?
My CRD's config, I've used Nifi + Keycloak with OIDC. Them I have not need to configure this parameters:
createCert: true
includeJKS: true
secretName: bpeadmin_secrets
So, it's my CRD's configuration of users and groups:
apiVersion: nifi.orange.com/v1alpha1
kind: NifiUser
metadata:
name: wanderson.pereira
namespace: nifi
spec:
identity: wanderson.pereira@mydomain.com.br
clusterRef:
name: nifi-cluster
namespace: nifi
createCert: false
---
apiVersion: nifi.orange.com/v1alpha1
kind: NifiUserGroup
metadata:
name: nifi-admin
namespace: nifi
spec:
clusterRef:
name: nifi-cluster
namespace: nifi
accessPolicies:
- type: global
action: read
resource: /flow
@wandersonpereira Now my cluster is up and running
*** With OIDC Enabled **** For Auth I have used the AWS Cognito (For OIDC) For ingress I have used the : Elastic Loadbalacer controller I am able to create the users and groups:
My Requirement is to make S2S with other standalone nifi instances and this nificluster.
but with OIDC I am forced to use ALB as only alb can offer the sticky sessions that are needed for OIDC.
Now the problem with alb is it terminated ssl at loadbalancer but i need that ssl so i have to go back to the nificluster without OIDC
** without OIDC *** Now when I remove the OIDC:
My cluster is up and running
But the problem here is : the users and groups are not synchronized to the nificluster
When we create users and groups with CRDs the creation happens in 2 stages
the first stage is : reconciliation and second is synchronization
the synchronization is not happening in my cluster as per nifikop logs Cluster is not ready: please have a look at the logs below.
2022-01-07T08:36:27+05:30 github.com/go-logr/zapr.(*zapLogger).Error
2022-01-07T08:36:27+05:30 /go/pkg/mod/github.com/go-logr/zapr@v0.2.0/zapr.go:132
2022-01-07T08:36:27+05:30 github.com/Orange-OpenSource/nifikop/pkg/nificlient.errorGetOperation
2022-01-07T08:36:27+05:30 /workspace/pkg/nificlient/common.go:38
2022-01-07T08:36:27+05:30 github.com/Orange-OpenSource/nifikop/pkg/nificlient.(*nifiClient).DescribeCluster
2022-01-07T08:36:27+05:30 /workspace/pkg/nificlient/system.go:30
2022-01-07T08:36:27+05:30 github.com/Orange-OpenSource/nifikop/pkg/nificlient.(*nifiClient).Build
2022-01-07T08:36:27+05:30 /workspace/pkg/nificlient/client.go:181
2022-01-07T08:36:27+05:30 github.com/Orange-OpenSource/nifikop/pkg/nificlient.NewFromConfig
2022-01-07T08:36:27+05:30 /workspace/pkg/nificlient/client.go:202
2022-01-07T08:36:27+05:30 github.com/Orange-OpenSource/nifikop/pkg/common.NewClusterConnection
2022-01-07T08:36:27+05:30 /workspace/pkg/common/common.go:54
2022-01-07T08:36:27+05:30 github.com/Orange-OpenSource/nifikop/pkg/clientwrappers/scale.EnsureRemovedNodes
2022-01-07T08:36:27+05:30 /workspace/pkg/clientwrappers/scale/scale.go:201
2022-01-07T08:36:27+05:30 github.com/Orange-OpenSource/nifikop/pkg/resources/nifi.(*Reconciler).Reconcile
2022-01-07T08:36:27+05:30 /workspace/pkg/resources/nifi/nifi.go:237
2022-01-07T08:36:27+05:30 github.com/Orange-OpenSource/nifikop/controllers.(*NifiClusterReconciler).Reconcile
2022-01-07T08:36:27+05:30 /workspace/controllers/nificluster_controller.go:131
2022-01-07T08:36:27+05:30 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
2022-01-07T08:36:27+05:30 /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.7.2/pkg/internal/controller/controller.go:263
2022-01-07T08:36:27+05:30 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
2022-01-07T08:36:27+05:30 /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.7.2/pkg/internal/controller/controller.go:235
2022-01-07T08:36:27+05:30 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.1
2022-01-07T08:36:27+05:30 /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.7.2/pkg/internal/controller/controller.go:198
2022-01-07T08:36:27+05:30 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1
2022-01-07T08:36:27+05:30 /go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:185
2022-01-07T08:36:27+05:30 k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1
2022-01-07T08:36:27+05:30 /go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:155
2022-01-07T08:36:27+05:30 k8s.io/apimachinery/pkg/util/wait.BackoffUntil
2022-01-07T08:36:27+05:30 /go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:156
2022-01-07T08:36:27+05:30 k8s.io/apimachinery/pkg/util/wait.JitterUntil
2022-01-07T08:36:27+05:30 /go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:133
2022-01-07T08:36:27+05:30 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext
2022-01-07T08:36:27+05:30 /go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:185
2022-01-07T08:36:27+05:30 k8s.io/apimachinery/pkg/util/wait.UntilWithContext
2022-01-07T08:36:27+05:30 /go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:99
2022-01-07T08:36:27+05:30 2022-01-07T03:06:27.255Z INFO controllers.NifiCluster Nodes unreachable, may still be starting up
2022-01-07T08:36:27+05:30 2022-01-07T03:06:27.308Z DEBUG controller-runtime.manager.events Normal {"object": {"kind":"NifiUser","namespace":"nifi","name":"sslnifi-controller.nifi.mgt.cluster.local","uid":"7a1717e4-c837-4503-b131-e3fb1f904638","apiVersion":"nifi.orange.com/v1alpha1","resourceVersion":"71815037"}, "reason": "ReconcilingCertificate", "message": "Reconciling certificate for nifi user sslnifi-controller.nifi.mgt.cluster.local"}
2022-01-07T08:36:27+05:30 2022-01-07T03:06:27.314Z DEBUG controller-runtime.manager.events Normal {"object": {"kind":"NifiUser","namespace":"nifi","name":"sslnifi-controller.nifi.mgt.cluster.local","uid":"7a1717e4-c837-4503-b131-e3fb1f904638","apiVersion":"nifi.orange.com/v1alpha1","resourceVersion":"71815037"}, "reason": "ReconciledCertificate", "message": "Reconciled certificate for nifi user sslnifi-controller.nifi.mgt.cluster.local"}
2022-01-07T08:36:27+05:30 2022-01-07T03:06:27.314Z INFO controllers.NifiUser Cluster is not ready yet, will wait until it is.
2022-01-07T08:36:27+05:30 2022-01-07T03:06:27.314Z DEBUG controller-runtime.manager.events Normal {"object": {"kind":"NifiUser","namespace":"nifi","name":"sslnifi-controller.nifi.mgt.cluster.local","uid":"7a1717e4-c837-4503-b131-e3fb1f904638","apiVersion":"nifi.orange.com/v1alpha1","resourceVersion":"71815037"}, "reason": "ReferenceClusterNotReady", "message": "The referenced cluster is not ready yet : sslnifi in sslnifi"}
2022-01-07T08:36:27+05:30 2022-01-07T03:06:27.323Z DEBUG controller-runtime.manager.events Normal {"object": {"kind":"NifiUser","namespace":"nifi","name":"sslnifi-1-node.sslnifi-headless.nifi.svc.cluster.local","uid":"3fb47db1-727e-40ca-b864-55c327f793d4","apiVersion":"nifi.orange.com/v1alpha1","resourceVersion":"71803986"}, "reason": "ReconcilingCertificate", "message": "Reconciling certificate for nifi user sslnifi-1-node.sslnifi-headless.nifi.svc.cluster.local"}
2022-01-07T08:36:27+05:30 2022-01-07T03:06:27.327Z DEBUG controller-runtime.manager.events Normal {"object": {"kind":"NifiUser","namespace":"nifi","name":"sslnifi-1-node.sslnifi-headless.nifi.svc.cluster.local","uid":"3fb47db1-727e-40ca-b864-55c327f793d4","apiVersion":"nifi.orange.com/v1alpha1","resourceVersion":"71803986"}, "reason": "ReconciledCertificate", "message": "Reconciled certificate for nifi user sslnifi-1-node.sslnifi-headless.nifi.svc.cluster.local"}
2022-01-07T08:36:27+05:30 2022-01-07T03:06:27.327Z INFO controllers.NifiUser Cluster is not ready yet, will wait until it is.
As per the logs when I removed the OIDC config from the Cluster, nifikop assuming the cluster is not ready yet. controllers.NifiUser Cluster is not ready yet, will wait until it is.
This is my Nificluster config:
apiVersion: nifi.orange.com/v1alpha1
kind: NifiCluster
metadata:
name: sslnifi
spec:
service:
headlessEnabled: true
annotations:
external-dns.alpha.kubernetes.io/ttl: "5"
zkAddress: "zookeeper.zookeeper.svc.cluster.local:2181"
zkPath: "/ssllnifi"
clusterImage: "apache/nifi:1.12.1"
oneNifiNodePerNode: false
managedAdminUsers:
- identity : "CN=admin" # I have named identity CN bcz the certificate will be styarting with CN so we need to mention it here to.
name: "admin"
propagateLabels: true
nifiClusterTaskSpec:
retryDurationMinutes: 10
readOnlyConfig:
nifiProperties:
webProxyHosts:
- sslnifi.mydomain.org:8443
nodeConfigGroups:
default_group:
isNode: true
storageConfigs:
- mountPath: "/opt/nifi/nifi-current/logs"
name: logs
pvcSpec:
accessModes:
- ReadWriteOnce
storageClassName: "gp2"
resources:
requests:
storage: 10Gi
- mountPath: "/opt/nifi/data"
name: data
pvcSpec:
accessModes:
- ReadWriteOnce
storageClassName: "gp2"
resources:
requests:
storage: 10Gi
- mountPath: "/opt/nifi/flowfile_repository"
name: flowfile-repository
pvcSpec:
accessModes:
- ReadWriteOnce
storageClassName: "gp2"
resources:
requests:
storage: 10Gi
- mountPath: "/opt/nifi/nifi-current/conf"
name: conf
pvcSpec:
accessModes:
- ReadWriteOnce
storageClassName: "gp2"
resources:
requests:
storage: 10Gi
- mountPath: "/opt/nifi/content_repository"
name: content-repository
pvcSpec:
accessModes:
- ReadWriteOnce
storageClassName: "gp2"
resources:
requests:
storage: 10Gi
- mountPath: "/opt/nifi/provenance_repository"
name: provenance-repository
pvcSpec:
accessModes:
- ReadWriteOnce
storageClassName: "gp2"
resources:
requests:
storage: 10Gi
serviceAccountName: "default"
resourcesRequirements:
limits:
cpu: "0.5"
memory: 2Gi
requests:
cpu: "0.5"
memory: 2Gi
nodes:
- id: 1
nodeConfigGroup: "default_group"
- id: 2
nodeConfigGroup: "default_group"
# - id: 3
# nodeConfigGroup: "default_group"
listenersConfig:
internalListeners:
- type: "https"
name: "https"
containerPort: 8443
- type: "cluster"
name: "cluster"
containerPort: 6007
- type: "s2s"
name: "s2s"
containerPort: 10000
sslSecrets:
tlsSecretName: "test-nifikop"
create: true
Have you tried without any OIDC, can you check if you are also facing same Issue. If we can Identify why nifikop is assuming the cluster is not ready I think we can solve this problem.
please have a look here: https://github.com/Orange-OpenSource/nifikop/issues/49 this is the issue i am facing now without OIDC scenario.
Hello @Sreenivas-Ratakonda .
Sorry, about time to response you! Your problem is solved?
Bug Report
After setting up the sslnifi cluster I found that the managed users are not getting created, as per the docs to login in to the Nifi cluster UI we need one admin user but that user is not getting created in the nificluster. As per the docs by default three groups gets created
managed admins, managed users, managed nodes
but for me the nifi user groups are not getting createdWhat did you do? I have created an sslnifi cluster.
What did you expect to see?
we expected see managed users to be created but those users are not created in Nifi Cluster. I have created an another user
bpeadmin
when I query nifikop it says user created but the user is not created in Nificluster. few Nifi user groups needs be created.What did you see instead? Under which circumstances?
Below we can see that there are no managed users created, which are mentioned in the Nifi Cluster config.
Here it says that bpeadmin user is created but i have added authorizer file there is no bpeadmin user created in there.
Users created in the Nifi Cluster config
authorizers.xml file in one of the nodes.
Detailed view at the bpeadmin user
No Nifi Groups Found
So to summarize there is a conflict between what we see in
k get nifiusers.nifi.orange.com -n nifi
andauthorizers.xml
one says the bpeadmin user created but the other one doesn't have the the bpeadmin user in authorizers.xmlSo over all the Users are not getting created in Nifi Cluster
Environment
nifikop version: Followed exact steps here:
https://orange-opensource.github.io/nifikop/docs/2_setup/1_getting_started
Kubernetes version information:
1.12.1
Possible Solution
Additional context Add any other context about the problem here.