search

cetic / helm-nifi

Helm Chart for Apache Nifi
Apache License 2.0
215 stars 228 forks source link

Breaks site-to-site test, when upgrade app version from 1.16.3 to 1.19.0 #281

Open dtrdnk opened 1 year ago

dtrdnk commented 1 year ago

Describe the bug Current app version 1.16.3 works fine, but if you upgrade to 1.19.0, then the automatic rotation of certificates breaks. This breaks also breaks the test-site-to-site

Version of Helm, Kubernetes and the Nifi chart: Helm : v3.10.2 Kubernetes: v1.25.3 NiFi chart: 1.1.3

What happened: NiFi server is still working with the old certificate, even if new cert exist in the store

What you expected to happen: NiFi server must use a new certificate on the fly without restart

How to reproduce it (as minimally and precisely as possible): Just upgrade image tag from 1.16.3 to 1.19.0 or 1.18.0

Anything else we need to know:

Here are some information that help troubleshooting: NiFi has autoreload function, but I don't know how to invoke it. By default autoreload set to false. Even if I set manualy autoreload to true, this is doesn't help.

Check if a pod is in error: There is no a pod error

jdesroch commented 1 year ago

This is most likely due to NiFi 1.19.0 docker base image changed to eclipse-temurin:jre-11 from openjdk:8-jre. This changed the cacerts location from /usr/local/openjdk-8/lib/security/cacerts to /opt/java/openjdk/lib/security/cacerts.

dtrdnk commented 1 year ago

This is most likely due to NiFi 1.19.0 docker base image changed to eclipse-temurin:jre-11 from openjdk:8-jre. This changed the cacerts location from /usr/local/openjdk-8/lib/security/cacerts to /opt/java/openjdk/lib/security/cacerts.

Maybe you are right. What we can to do with this problem?

wknickless commented 1 year ago

This is most likely due to NiFi 1.19.0 docker base image changed to eclipse-temurin:jre-11 from openjdk:8-jre. This changed the cacerts location from /usr/local/openjdk-8/lib/security/cacerts to /opt/java/openjdk/lib/security/cacerts.

Yes, but with cert-manager enabled the chart overrides that default by setting the nifi.security.truststore configuration property. And it also sets the nifi.security.autoreload.enabled and nifi.security.autoreload.interval properties per the documentation (https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#security_configuration).

We are also interested in upgrading to NiFi 1.18 or 1.19 for other reasons, so I'll get to work replicating the problem.

wknickless commented 1 year ago

Confirmed that the certificate rotation broke between 1.16.3 and 1.17.0. Suspicious of this commit: https://github.com/apache/nifi/commit/4b655ecb688685016f67fba7a9386683f8429df6

banzo commented 1 year ago

same for 1.20, see #294

banzo commented 1 year ago

@wknickless @dtrdnk #294 fixes the issue and bumps NiFi to 1.20.

I'm happy to merge the PR but I would be more at ease with more eyes on the changes.

nathluu commented 1 year ago

Hi @banzo, in my PR #294, I just reatart nifi pod to make it reload new keystore, it seems the autoreload feature is broken and has not been fixed yet

dtrdnk commented 1 year ago

@wknickless @dtrdnk #294 fixes the issue and bumps NiFi to 1.20.

I'm happy to merge the PR but I would be more at ease with more eyes on the changes.

Good news! I wiil wait merge of PR 294. And then I will merge all commits from master into my PR

dtrdnk commented 1 year ago

Hi @banzo, in my PR #294, I just reatart nifi pod to make it reload new keystore, it seems the autoreload feature is broken and has not been fixed yet

You are right. Autoreload function is broken in new versions of Docker image, and I don't know how to fix this problem.

banzo commented 1 year ago

@dtrdnk is one of those issues describing the problem or do we need to create a new one?

https://issues.apache.org/jira/browse/NIFI-10425?jql=project%20%3D%20NIFI%20AND%20text%20~%20autoreload

dtrdnk commented 1 year ago

@dtrdnk is one of those issues describing the problem or do we need to create a new one?

https://issues.apache.org/jira/browse/NIFI-10425?jql=project%20%3D%20NIFI%20AND%20text%20~%20autoreload

Thank you! This issue NIFI-10425 looks like our problem. I make research by the Jetty problem, and get some facts:

I think we should just wait for the NIFI-11518 task to be released.

nathluu commented 1 year ago

Hi @dtrdnk, I tested locally on windows (no symlink) machine and the autoreload feature has also broken since v1.17.0. I created NIFI-11536 for tracking this issue. The feature works for 1.16.3 so it clearly is not the issue with jetty