authorizers.xml always overwritten

M9T commented 4 years ago

I wanted to change the authorizers.xml based on my needs.

What I did:

Run docker run command

docker run --name nifi-registry \
  -v nifi-registry-certs:/opt/certs \
  -v nifi-registry-conf:/opt/nifi-registry/nifi-registry-0.5.0/conf \
  -p 18443:18443 \
  -e 'NIFI_REGISTRY_SECURITY_KEYSTORE=/opt/certs/keystore.jks' \
  -e 'NIFI_REGISTRY_SECURITY_KEYSTOREtYPE=JKS' \
  -e 'NIFI_REGISTRY_SECURITY_KEYSTOREpASSWD=pw' \
  -e 'NIFI_REGISTRY_SECURITY_TRUSTSTORE=/opt/certs/truststore.jks' \
  -e 'NIFI_REGISTRY_SECURITY_TRUSTSTOREtYPE=JKS' \
  -e 'NIFI_REGISTRY_SECURITY_TRUSTSTOREpASSWD=pw' \
  -e 'NIFI_REGISTRY_WEB_HTTP_HOST=' \
  -e 'NIFI_REGISTRY_WEB_HTTP_PORT=' \
  -e 'NIFI_REGISTRY_WEB_HTTPS_HOST=0.0.0.0' \
  -e 'NIFI_REGISTRY_WEB_HTTPS_PORT=18443' \
  -e 'INITIAL_ADMIN_IDENTITY=CN=...' \
  -e 'NIFI_REGISTRY_SECURITY_IDENTITY_PROVIDER=ldap-identity-provider' \
  -e 'NIFI_REGISTRY_SECURITY_NEEDcLIENTaUTH=false' \
  -e LDAP_URL='ldap://...' \
  -e LDAP_MANAGER_DN='cn=...' \
  -e LDAP_MANAGER_PASSWORD='pw' \
  -e LDAP_AUTHENTICATION_STRATEGY='SIMPLE' \
  -e LDAP_USER_SEARCH_BASE='dc=nifi,dc=com' \
  -e LDAP_USER_SEARCH_FILTER='cn={0}' \
  -e 'FLOW_PROVIDER=git' \
  -e 'GIT_REMOTE_URL=git@github...registry.git' \
  -e 'FLOW_PROVIDER_GIT_FLOW_STORAGE_DIRECTORY=/opt/nifi-registry/flow-storage-git' \
  -e 'GIT_CONFIG_USER_NAME=name' \
  -e 'GIT_CONFIG_USER_EMAIL=mail' \
  -e 'SSH_PRIVATE_KEY='$(base64 -w 0 < ~/.ssh/id_rsa) \
  -e 'SSH_KNOWN_HOSTS='$(base64 -w 0 < ~/.ssh/known_hosts) \
  -e 'SSH_PRIVATE_KEY_PASSPHRASE=' \
  -d \
  michalklempa/nifi-registry:latest

I checked the authorizers.xml file (and figured out that I need to modify some authz stuff*)
Did the changes in authorizers.xml
Restart the container
Changes are gone :-(

I haven't tested all files in the conf volume, but I guess it is a general overwriting.

What I would like to set:

authorizers.xml

ldap-user-group-provider
composite-configurable-user-group-provider
composite-configurable-user-group-provider

nifi-registry.properties file

nifi.security.identity.mapping.pattern.dn=^cn=(.?),o=(.?),dc=(.?),dc=(.?)$
nifi.security.identity.mapping.value.dn=$1

M9T commented 4 years ago

authorizers.xml

50% covered by replacing the volume with -v $HOME/nifi/registry/conf/authorizers.xml:/opt/nifi-registry/nifi-registry-0.5.0/conf/authorizers.xml \ and deleting the INITIAL_ADMIN_IDENTITY env.

I tried to remove all NIFI_REGISTRY envs and add -v $HOME/nifi/registry/conf/nifi-registry.properties:/opt/nifi-registry/nifi-registry-0.5.0/conf/nifi-registry.properties \ but it fails:

Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.apache.nifi.registry.security.authentication.IdentityProvider]: Factory method 'getIdentityProvider' threw exception; nested exception is java.lang.Exception: The specified login identity provider 'ldap-identity-provider' could not be found. at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:185) at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:622) ... 57 common frames omitted Caused by: java.lang.Exception: The specified login identity provider 'ldap-identity-provider' could not be found. at org.apache.nifi.registry.security.authentication.IdentityProviderFactory.getIdentityProvider(IdentityProviderFactory.java:128) at org.apache.nifi.registry.security.authentication.IdentityProviderFactory$$EnhancerBySpringCGLIB$$1a9d8ad4.CGLIB$getIdentityProvider$0(<generated>) at org.apache.nifi.registry.security.authentication.IdentityProviderFactory$$EnhancerBySpringCGLIB$$1a9d8ad4$$FastClassBySpringCGLIB$$f58f9651.invoke(<generated>) at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:244) at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:363) at org.apache.nifi.registry.security.authentication.IdentityProviderFactory$$EnhancerBySpringCGLIB$$1a9d8ad4.getIdentityProvider(<generated>) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:154) ... 58 common frames omitted 2020-01-23 21:16:03,202 INFO [Thread-1] org.apache.nifi.registry.NiFiRegistry Initiating shutdown of Jetty web server... 2020-01-23 21:16:03,209 INFO [Thread-1] o.eclipse.jetty.server.AbstractConnector Stopped ServerConnector@42f93a98{SSL,[ssl, http/1.1]}{0.0.0.0:18443} 2020-01-23 21:16:03,209 INFO [Thread-1] org.eclipse.jetty.server.session node0 Stopped scavenging Received trapped signal, beginning shutdown...

michalklempa commented 4 years ago

Hi, as you found out, the authorizers.xml are templated from env vars. Corresponding line in shell script is: https://github.com/michalklempa/docker-nifi-registry/blob/develop/sh/start.sh

if [[ -n "${INITIAL_ADMIN_IDENTITY}" ]]; then
    /usr/local/bin/dockerize -template ${PROJECT_TEMPLATE_DIR}/authorizers.xml.gotemplate:${PROJECT_CONF_DIR}/authorizers.xml
fi

So not setting INITIAL_ADMIN_IDENTITY should do the trick of not replacing the file again and again.

Regarding

<accessPolicyProvider>
  <property name="User Group Provider">composite-configurable-user-group-provider

I can see, that I haven't done the templating of this file yet. If you can workaround by providing the file from outside until I put a new feature, would be fine.

Ad nifi-registry.properties file, the https://nifi.apache.org/docs/nifi-registry-docs/html/administration-guide.html#identity-mapping-properties should be able to set by: NIFI_REGISTRY_SECURITY_IDENTITY_MAPPING_PATTERN_DN

see: https://github.com/michalklempa/docker-nifi-registry/blob/develop/templates/nifi-registry.properties.gotemplate#L86

and NIFI_REGISTRY_SECURITY_IDENTITY_MAPPING_VALUE_DN

see: https://github.com/michalklempa/docker-nifi-registry/blob/develop/templates/nifi-registry.properties.gotemplate#L90

If you removed all NIFI_REGISTRY env vars and even after that the file nifi-registry.conf is overwritten, its a bug. It shouldnt be overwritten: https://github.com/michalklempa/docker-nifi-registry/blob/develop/sh/start.sh#L77

But the keys in nifi-registry.conf you need, are available, but are named differently. You stated:

nifi.security.identity.mapping.pattern.dn=^cn=(.?),o=(.?),dc=(.?),dc=(.?)$ nifi.security.identity.mapping.value.dn=$1 but the real ones are: nifi.registry.security.identity.mapping.pattern.dn=^cn=(.?),o=(.?),dc=(.?),dc=(.?)$ nifi.registry.security.identity.mapping.value.dn=$1

Let me know how it goes.

michalklempa commented 4 years ago

Note to myself, see #20 , templating should be re-worked. More and more people like to provide configuration k8s way by injecting the files from config-maps.

michalklempa commented 4 years ago

Hi @M9T, I have created plain flavor of images. Tagged with suffix -plain. These images run under root user (no UID:GID added) and no templating is done at all.

Try: https://hub.docker.com/layers/michalklempa/nifi-registry/0.5.0-plain/images/sha256-48e7ac738ae19146cd68672ce414dafae29937edbcd985a10fa3712c14a334f9?context=repo

michalklempa / docker-nifi-registry

authorizers.xml always overwritten #19