Kerberos - renew_lifetime setting make ranger service unable to start

[ChristopheLRTE]

Hello all !

We encountered an issue with the renew_lifetime = 7d setting in krb5.conf (see conf. files below). Indeed, when we add first of all this setting in krb5.conf (without update kdc.conf), the ranger service doesn't start (please see catalina.out logs below too). We hoped that adding the setting default_principal_flags = +renewable in kdc.conf would make the things working well, but we finally observed the same behaviour.

So as a workaround, we comment the renew_lifetime = 7d setting in krb5.conf and the ranger service started (after a manual restart ranger service command) !

Would you know why this behaviour and how to configure a renewable kerberos token please ?

krb5.conf.j2

includedir /etc/krb5.conf.d/

[logging]
 default = FILE:/var/log/krb5libs.log
 kdc = FILE:/var/log/krb5kdc.log
 admin_server = FILE:/var/log/kadmind.log

[libdefaults]
 dns_lookup_realm = false
 ticket_lifetime = 24h
 forwardable = true
 rdns = false
 pkinit_anchors = FILE:/etc/pki/tls/certs/ca-bundle.crt
 default_realm = {{ realm }}
 default_ccache_name = /tmp/krb5cc_%{uid}
 canonicalize = true
 renew_lifetime = 7d
 kdc_timeout = 5000
 dns_lookup_kdc = false

[realms]
{{ realm }} = {
  kdc = {{ kerberos_server }}
  admin_server = {{ kerberos_server }}
  default_domain = {{ domain }}
}

[domain_realm]
.{{ domain }} = {{ realm }}
{{ domain }} = {{ realm }}

kdc.conf.j2

includedir /etc/krb5.conf.d/

[libdefaults]
 default_realm = {{ realm }}

[logging]
 default = FILE:/var/log/krb5libs.log
 kdc = FILE:/var/log/krb5kdc.log
 admin_server = FILE:/var/log/kadmind.log

[realms]
{{ realm }} = {
  kdc = {{ kerberos_server }}
  admin_server = {{ kerberos_server }}
  default_domain = {{ domain }}
  default_principal_flags = +renewable
  database_module = openldap_ldapconf
}

[dbdefaults]
ldap_kerberos_container_dn = cn=krbContainer,{{ ldap_suffix }}

[dbmodules]
openldap_ldapconf = {
  db_library = kldap

  # if either of these is false, then the ldap_kdc_dn needs to
  # have write access
  disable_last_success = true
  disable_lockout  = true

  # this object needs to have read rights on
  # the realm container, principal container and realm sub-trees
  ldap_kdc_dn = {{ ldap_admin_dn }}

  # this object needs to have read and write rights on
  # the realm container, principal container and realm sub-trees
  ldap_kadmind_dn = {{ ldap_admin_dn }}

  ldap_service_password_file = {{ ldap_password_file }}
  ldap_servers = ldapi:///
  ldap_conns_per_server = 5
}

catalina.out logs

SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
juin 03, 2022 4:00:00 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFOS: Deriving webapp folder from catalina.base property. folder=/opt/tdp/ranger-2.0.1-TDP-0.1.0-SNAPSHOT-admin/ews/webapp
juin 03, 2022 4:00:00 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFOS: Webapp file =/opt/tdp/ranger-2.0.1-TDP-0.1.0-SNAPSHOT-admin/ews/webapp, webAppName = /
juin 03, 2022 4:00:00 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFOS: Adding webapp [/] = path [/opt/tdp/ranger-2.0.1-TDP-0.1.0-SNAPSHOT-admin/ews/webapp] .....
juin 03, 2022 4:00:00 PM org.apache.catalina.core.StandardContext setPath
AVERTISSEMENT: A context path must either be an empty string or start with a '/' and do not end with a '/'. The path [/] does not meet these criteria and has been changed to []
juin 03, 2022 4:00:00 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFOS: Finished init of webapp [/] = path [/opt/tdp/ranger-2.0.1-TDP-0.1.0-SNAPSHOT-admin/ews/webapp].
juin 03, 2022 4:00:00 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFOS: Provided Kerberos Credential : Principal = rangeradmin/master-03.applispfref.sipfref.local@BDP01.APPLISPFREF.SIPFREF.LOCAL and Keytab = /etc/security/keytabs/rangeradmin.service.keytab
juin 03, 2022 4:00:00 PM org.apache.ranger.server.tomcat.EmbeddedServer start
GRAVE: Tomcat Server failed to start:java.io.IOException: Login failure for rangeradmin/master-03.applispfref.sipfref.local@BDP01.APPLISPFREF.SIPFREF.LOCAL from keytab /etc/security/keytabs/rangeradmin.service.keytab
java.io.IOException: Login failure for rangeradmin/master-03.applispfref.sipfref.local@BDP01.APPLISPFREF.SIPFREF.LOCAL from keytab /etc/security/keytabs/rangeradmin.service.keytab
        at org.apache.hadoop.security.SecureClientLogin.loginUserFromKeytab(SecureClientLogin.java:71)
        at org.apache.ranger.server.tomcat.EmbeddedServer.start(EmbeddedServer.java:249)
        at org.apache.ranger.server.tomcat.EmbeddedServer.main(EmbeddedServer.java:69)
Caused by: javax.security.auth.login.LoginException: Message stream modified (41)
        at com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:810)
        at com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:618)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at javax.security.auth.login.LoginContext.invoke(LoginContext.java:755)
        at javax.security.auth.login.LoginContext.access$000(LoginContext.java:195)
        at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682)
        at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)
        at javax.security.auth.login.LoginContext.login(LoginContext.java:587)
        at org.apache.hadoop.security.SecureClientLogin.loginUserFromKeytab(SecureClientLogin.java:68)
        ... 2 more
Caused by: KrbException: Message stream modified (41)
        at sun.security.krb5.KrbKdcRep.check(KrbKdcRep.java:103)
        at sun.security.krb5.KrbAsRep.decrypt(KrbAsRep.java:159)
        at sun.security.krb5.KrbAsRep.decryptUsingKeyTab(KrbAsRep.java:121)
        at sun.security.krb5.KrbAsReqBuilder.resolve(KrbAsReqBuilder.java:310)
        at sun.security.krb5.KrbAsReqBuilder.action(KrbAsReqBuilder.java:498)
        at com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:782)

[ChristopheLRTE]

Hello, sorry to get back to you. Just to know if someone would have a solution please ?

Thank you all for your help !

[rpignolet]

Hi, PR https://github.com/TOSIT-IO/tdp-collection-prerequisites/pull/10 added LDAP and Kerberos deployment, ansible-ldap-kerberos will be removed (see TOSIT-IO/tdp-collection-extras#53) and I didn't change the actual behaviour i.e. the renew_lifetime = 7d is still present.

I transfer this issue to https://github.com/TOSIT-IO/tdp-collection-prerequisites repository and we will investigate.

[gboutry]

Are you encountering this issue while using tdp-getting-started with vagrant environment, or using your own environment ? If the later is the case, could provide your java vendor / version ? Example:

java -version
openjdk version "1.8.0_342"
OpenJDK Runtime Environment (build 1.8.0_342-b07)
OpenJDK 64-Bit Server VM (build 25.342-b07, mixed mode)

[gboutry]

I have been able to reproduce the issue. The root cause is because of how rangeradmin asks for tickets and a misconfiguration on the kdc. By default, the max renewable time is set to 0, and we don't configure it on the realm.

From here, you have 2 options to fix this situation:

If you can wipe your installation

Configure your kdc.conf as follow:

[realms]
{{ realm }} = {
  kdc = {{ kerberos_server }}
  admin_server = {{ kerberos_server }}
  default_domain = {{ domain }}
  default_principal_flags = +renewable
+ max_renewable_life = 7d # or any duration
  database_module = openldap_ldapconf
}

And then redeploy everything.

If you cannot wipe your installation but have access to kadmin

You still need to configure the kdc, if you want any new principal created renewable.

Then, you need to modify your existing principals to set them the max renewable time:

kadmin -p <admin_principal> -w <admin_password> -q "modprinc -maxrenewlife 7day krbtgt/BDP01.APPLISPFREF.SIPFREF.LOCAL"
kadmin -p <admin_principal> -w <admin_password> -q "modprinc -maxrenewlife 7day rangeradmin/master-03.applispfref.sipfref.local@BDP01.APPLISPFREF.SIPFREF.LOCAL"

[ChristopheLRTE]

Thank you very much @gboutry. We will try it 👍

Cheers

[gboutry]

This issue was closed automatically by merging #12, re-opening and waiting on @ChristopheLRTE feedback

[gboutry]

Considering this issue as solved since there is no feedback.