search

konpyutaika / nifikop

The NiFiKop NiFi Kubernetes operator makes it easy to run Apache NiFi on Kubernetes. Apache NiFI is a free, open-source solution that support powerful and scalable directed graphs of data routing, transformation, and system mediation logic.
https://konpyutaika.github.io/nifikop/
Apache License 2.0
122 stars 39 forks source link

Add settings for doing secure auth LDAPS/AD #391

Closed wrender closed 3 months ago

wrender commented 4 months ago
Q A
Bug fix? yes
New feature? no yes
API breaks? no yes
Deprecations? no yes
Related tickets fixes #X, partially #Y, mentioned in #Z
License Apache 2.0

What's in this PR?

Adding settings required to do secure LDAP with nifi

Why?

Missing settings for doing secure LDAP

Checklist

To Do

mh013370 commented 4 months ago

Would you mind updating the LDAP documentation here for the new configurable fields?

https://github.com/konpyutaika/nifikop/blob/master/site/docs/5_references/1_nifi_cluster/1_nifi_cluster.md?plain=1#L189-L196

wrender commented 4 months ago

Would you mind updating the LDAP documentation here for the new configurable fields?

https://github.com/konpyutaika/nifikop/blob/master/site/docs/5_references/1_nifi_cluster/1_nifi_cluster.md?plain=1#L189-L196

Ok. Done

wrender commented 4 months ago

For some reason when I define these new settings in a cluster crd yaml file, and then deploy a cluster they don't take affect in the container. Am I missing something? For example:

ldapConfiguration
  enabled: true
  tlsTruststore:  /some/path

Then if I exec into the container, and cat conf/login-identity-providers.xml , The value for that line item is empty.

mh013370 commented 4 months ago

Just one minor doc suggestion. otherwise it looks good to me

mh013370 commented 4 months ago

For some reason when I define these new settings in a cluster crd yaml file, and then deploy a cluster they don't take affect in the container. Am I missing something? For example:

ldapConfiguration
  enabled: true
  tlsTruststore:  /some/path

Then if I exec into the container, and cat conf/login-identity-providers.xml , The value for that line item is empty.

Just double checking you've applied the new CRDs here and the updated operator?

wrender commented 4 months ago

For some reason when I define these new settings in a cluster crd yaml file, and then deploy a cluster they don't take affect in the container. Am I missing something? For example:

ldapConfiguration
  enabled: true
  tlsTruststore:  /some/path

Then if I exec into the container, and cat conf/login-identity-providers.xml , The value for that line item is empty.

Just double checking you've applied the new CRDs here and the updated operator?

I deleted the CRDs and re-created them with the helm chart. Maybe I'm not updating the operator correctly? How would I update that?

mh013370 commented 4 months ago

For some reason when I define these new settings in a cluster crd yaml file, and then deploy a cluster they don't take affect in the container. Am I missing something? For example:

ldapConfiguration
  enabled: true
  tlsTruststore:  /some/path

Then if I exec into the container, and cat conf/login-identity-providers.xml , The value for that line item is empty.

Just double checking you've applied the new CRDs here and the updated operator?

I deleted the CRDs and re-created them with the helm chart. Maybe I'm not updating the operator correctly? How would I update that?

There's a thread in slack where folks talk about that: https://konpytika.slack.com/archives/C0362VBRM24/p1702911625342399

TLDR is that the helm client doesn't do it for you, but tools like ArgoCD/Flux will do it for you.

https://helm.sh/docs/chart_best_practices/custom_resource_definitions/#some-caveats-and-explanations

wrender commented 4 months ago

For some reason when I define these new settings in a cluster crd yaml file, and then deploy a cluster they don't take affect in the container. Am I missing something? For example:

ldapConfiguration
  enabled: true
  tlsTruststore:  /some/path

Then if I exec into the container, and cat conf/login-identity-providers.xml , The value for that line item is empty.

Just double checking you've applied the new CRDs here and the updated operator?

I deleted the CRDs and re-created them with the helm chart. Maybe I'm not updating the operator correctly? How would I update that?

There's a thread in slack where folks talk about that: https://konpytika.slack.com/archives/C0362VBRM24/p1702911625342399

TLDR is that the helm client doesn't do it for you, but tools like ArgoCD/Flux will do it for you.

https://helm.sh/docs/chart_best_practices/custom_resource_definitions/#some-caveats-and-explanations

I manually deleted the CRDs, and confirmed they are updated after re-installing the nifikop helm chart. But still, when I deploy a nifi cluster it doesn't seem to add the ldap settings. Seems like it must be something else then.

juldrixx commented 4 months ago

For some reason when I define these new settings in a cluster crd yaml file, and then deploy a cluster they don't take affect in the container. Am I missing something? For example:

ldapConfiguration
  enabled: true
  tlsTruststore:  /some/path

Then if I exec into the container, and cat conf/login-identity-providers.xml , The value for that line item is empty.

Just double checking you've applied the new CRDs here and the updated operator?

I deleted the CRDs and re-created them with the helm chart. Maybe I'm not updating the operator correctly? How would I update that?

There's a thread in slack where folks talk about that: https://konpytika.slack.com/archives/C0362VBRM24/p1702911625342399 TLDR is that the helm client doesn't do it for you, but tools like ArgoCD/Flux will do it for you. https://helm.sh/docs/chart_best_practices/custom_resource_definitions/#some-caveats-and-explanations

I manually deleted the CRDs, and confirmed they are updated after re-installing the nifikop helm chart. But still, when I deploy a nifi cluster it doesn't seem to add the ldap settings. Seems like it must be something else then.

Did you build an image of the operator with your change? Or are you using the 1.7.0? Or are you running the code locally?

wrender commented 4 months ago

For some reason when I define these new settings in a cluster crd yaml file, and then deploy a cluster they don't take affect in the container. Am I missing something? For example:

ldapConfiguration
  enabled: true
  tlsTruststore:  /some/path

Then if I exec into the container, and cat conf/login-identity-providers.xml , The value for that line item is empty.

Just double checking you've applied the new CRDs here and the updated operator?

I deleted the CRDs and re-created them with the helm chart. Maybe I'm not updating the operator correctly? How would I update that?

There's a thread in slack where folks talk about that: https://konpytika.slack.com/archives/C0362VBRM24/p1702911625342399 TLDR is that the helm client doesn't do it for you, but tools like ArgoCD/Flux will do it for you. https://helm.sh/docs/chart_best_practices/custom_resource_definitions/#some-caveats-and-explanations

I manually deleted the CRDs, and confirmed they are updated after re-installing the nifikop helm chart. But still, when I deploy a nifi cluster it doesn't seem to add the ldap settings. Seems like it must be something else then.

Did you build an image of the operator with your change? Or are you using the 1.7.0? Or are you running the code locally?

Running the code locally. I git cloned my fork, that has the login providers changes for ldap, and then do a helm install of nifikop from that local folder.

juldrixx commented 4 months ago

For some reason when I define these new settings in a cluster crd yaml file, and then deploy a cluster they don't take affect in the container. Am I missing something? For example:

ldapConfiguration
  enabled: true
  tlsTruststore:  /some/path

Then if I exec into the container, and cat conf/login-identity-providers.xml , The value for that line item is empty.

Just double checking you've applied the new CRDs here and the updated operator?

I deleted the CRDs and re-created them with the helm chart. Maybe I'm not updating the operator correctly? How would I update that?

There's a thread in slack where folks talk about that: https://konpytika.slack.com/archives/C0362VBRM24/p1702911625342399 TLDR is that the helm client doesn't do it for you, but tools like ArgoCD/Flux will do it for you. https://helm.sh/docs/chart_best_practices/custom_resource_definitions/#some-caveats-and-explanations

I manually deleted the CRDs, and confirmed they are updated after re-installing the nifikop helm chart. But still, when I deploy a nifi cluster it doesn't seem to add the ldap settings. Seems like it must be something else then.

Did you build an image of the operator with your change? Or are you using the 1.7.0? Or are you running the code locally?

Running the code locally. I git cloned my fork, that has the login providers changes for ldap, and then do a helm install of nifikop from that local folder.

If you didn't build an image of your code to use in your deployment, it won't work. It will just deploy the latest release of the operator. If you want to run the code locally, you need to use a tool like telepresence.

wrender commented 4 months ago

For some reason when I define these new settings in a cluster crd yaml file, and then deploy a cluster they don't take affect in the container. Am I missing something? For example:

ldapConfiguration
  enabled: true
  tlsTruststore:  /some/path

Then if I exec into the container, and cat conf/login-identity-providers.xml , The value for that line item is empty.

Just double checking you've applied the new CRDs here and the updated operator?

I deleted the CRDs and re-created them with the helm chart. Maybe I'm not updating the operator correctly? How would I update that?

There's a thread in slack where folks talk about that: https://konpytika.slack.com/archives/C0362VBRM24/p1702911625342399 TLDR is that the helm client doesn't do it for you, but tools like ArgoCD/Flux will do it for you. https://helm.sh/docs/chart_best_practices/custom_resource_definitions/#some-caveats-and-explanations

I manually deleted the CRDs, and confirmed they are updated after re-installing the nifikop helm chart. But still, when I deploy a nifi cluster it doesn't seem to add the ldap settings. Seems like it must be something else then.

Did you build an image of the operator with your change? Or are you using the 1.7.0? Or are you running the code locally?

Running the code locally. I git cloned my fork, that has the login providers changes for ldap, and then do a helm install of nifikop from that local folder.

If you didn't build an image of your code to use in your deployment, it won't work. It will just deploy the latest release of the operator. If you want to run the code locally, you need to use a tool like telepresence.

Ok. Thanks for the information @juldrixx . I'm new to operators in Kubernetes so I will have to spend some time learning this. Is there any documentation on how I would go about the building of a custom image to test with the deployment? I don't know I want to introduce yet another tool like telepresence.

juldrixx commented 4 months ago

For some reason when I define these new settings in a cluster crd yaml file, and then deploy a cluster they don't take affect in the container. Am I missing something? For example:

ldapConfiguration
  enabled: true
  tlsTruststore:  /some/path

Then if I exec into the container, and cat conf/login-identity-providers.xml , The value for that line item is empty.

Just double checking you've applied the new CRDs here and the updated operator?

I deleted the CRDs and re-created them with the helm chart. Maybe I'm not updating the operator correctly? How would I update that?

There's a thread in slack where folks talk about that: https://konpytika.slack.com/archives/C0362VBRM24/p1702911625342399 TLDR is that the helm client doesn't do it for you, but tools like ArgoCD/Flux will do it for you. https://helm.sh/docs/chart_best_practices/custom_resource_definitions/#some-caveats-and-explanations

I manually deleted the CRDs, and confirmed they are updated after re-installing the nifikop helm chart. But still, when I deploy a nifi cluster it doesn't seem to add the ldap settings. Seems like it must be something else then.

Did you build an image of the operator with your change? Or are you using the 1.7.0? Or are you running the code locally?

Running the code locally. I git cloned my fork, that has the login providers changes for ldap, and then do a helm install of nifikop from that local folder.

If you didn't build an image of your code to use in your deployment, it won't work. It will just deploy the latest release of the operator. If you want to run the code locally, you need to use a tool like telepresence.

Ok. Thanks for the information @juldrixx . I'm new to operators in Kubernetes so I will have to spend some time learning this. Is there any documentation on how I would go about the building of a custom image to test with the deployment? I don't know I want to introduce yet another tool like telepresence.

You can find it here but it doesn't mention telepresence.

mh013370 commented 3 months ago

LGTM