IAM on Nifi

[banzo]

Add IAM on the Nifi service

[alexnuttinck]

Now, that we can deploy fadi with ssl, we need to connect nifi to openldap.

[AyadiAmen]

In order to create a secure NiFi cluster, we need a keystore and truststore and a client certificate that we can load in our browser to access the UI.

NIFI comes with The Apache NiFi Toolkit which is an additional artifact produced when building NiFi, and it is used to generate the keystore and truststore as well as a client cert for our browser, the toolkit path inside the pod is /opt/nifi/nifi-toolkit-current/. We can do that by running the following command:

/opt/nifi/nifi-toolkit-current/bin/tls-toolkit.sh standalone -n 'localhost' -C 'CN=bbende, OU=ApacheNiFi' -o '/opt/nifi/nifi-current/conf/'

• standalone: The TLS toolkit has two modes, client/server and standalone
• -n ‘localhost’ tells the toolkit that we want to run two NiFi instances on localhost
• -C specifies the DN of the client certificate that will be generated
• -o specifies the directory to use for the output.

Then we need to change a few properties in the file nifi.properties and set the following to:

nifi.state.management.embedded.zookeeper.start=true

nifi.cluster.protocol.is.secure=true
nifi.cluster.is.node=true

nifi.zookeeper.connect.string=localhost:2181

and set the followin properties to:

nifi.security.keystore=./conf/keystore.jks
nifi.security.keystoreType=jks
nifi.security.keystorePasswd=Eg3Laf9YaSk7D3dNChu+OYEbB7AYUz4vUOK+LvT14I4
nifi.security.keyPasswd=Eg3Laf9YaSk7D3dNChu+OYEbB7AYUz4vUOK+LvT14I4
nifi.security.truststore=./conf/truststore.jks
nifi.security.truststoreType=jks
nifi.security.truststorePasswd=eJ3aBnluTrN1tuJpNZovGULN4GRiVXRwf0K9E2/Krj0

where nifi.security.keystore and nifi.security.truststore are the paths where the keystore.jks and truststore.jks are going to be generated.

The nifi.security.keystorePasswd and nifi.security.truststorePasswd are the passwords of the keystore.jks and truststore.jks that we are going to generate.

For this to work in a helm chart all the process must take place at the creation of the pod.

To edit the file nifi.properties you have to edit the file ~/helm-nifi/conf/nifi.propeerties in the helm chart.

We're going to have to edit some other config files such as state-management.xml and authorizers.xml which we can do in the helm chart ~/helm-nifi/conf/.

If we don't pass the correct password it will raise the following error (the app-log logs ) and the pod won't launch:

error: java.io.IOException: Keystore was tampered with, or password was incorrect

For more info you can see these doc/tutorial: apache nifi authorization and multi tenancy -- NiFi and SSL -- TLS with nifi-toolkit

Potential solutions

1. extract and change the generated passwords.

One way to get the newly generated password is from the generated nifi.properties file. those commands will extract the passwords nifi.security.keystorePasswd and nifi.security.truststorePasswd from the newly created nifi.properties, and create the new files keystorePasswd.password and truststorePasswd.password.

grep -o 'nifi.security.keystorePasswd=.*' ./conf/localhost/nifi.properties | sed 's/nifi.security.keystorePasswd=//g' > keystorePasswd.password

grep -o 'nifi.security.truststorePasswd=.*' ./conf/localhost/nifi.properties | sed 's/nifi.security.truststorePasswd=//g' > truststorePasswd.password

Using the previously created password files, these commands change the passwords for the keystore.jks and truststore.jks to keystorePasswdfadi and truststorePasswdfadi respectively:

cat /opt/nifi/nifi-current/keystorePasswd.password | keytool -storepasswd -new keystorePasswdfadi -keystore ./conf/localhost/keystore.jks

cat /opt/nifi/nifi-current/truststorePasswd.password | keytool -storepasswd -new truststorePasswdfadi -keystore ./conf/localhost/truststore.jks

so that we can pass the new passwords in the nifi.properties .

to make this processus take place at the creation of the pod and before launching the server, you can pass them inside the StatefulSet.yaml file of the helm chart.

doing that we will no longer have the error:

error: java.io.IOException: Keystore was tampered with, or password was incorrect

but the pod is still not running correctly giving this error:

org.apache.nifi.web.server.JettyServer Failed to start web server... shutting down.
org.apache.nifi.web.NiFiCoreException: Unable to start Flow Controller.

Probably more configuration is required for this to work.

In this branch you can find the files with the mentioned edits following this commit.

2. Custom NiFi Docker Image

A second way for doing this is by using a custom nifi docker image following this tutorial.

[alexnuttinck]

https://github.com/cetic/helm-nifi/issues/45 will now be used to follow this issue.

[AyadiAmen]

This issus is solved as nifi is connected to ldap now, i'm closing this issue.