search

pingidentity / ldapsdk

UnboundID LDAP SDK for Java
Other
334 stars 81 forks source link

JAAS login context for GSSAPI authentication Exception on high load to AD/LDS #85

Open fnikhil-clgx opened 4 years ago

fnikhil-clgx commented 4 years ago

We have a SpringBoot 2.x REST API Service that is used do to various things against a AD/LDS Directory Server:

1. getUsersByEmail() given a valid email address of a user in AD/LDS
    returns the associted user record in AD/LDS
    read-only operation on AD/LDS
2. unsuspendPassword() given a valid user id of a user in AD/LDS
    unsuspends the associted user record in AD/LDS
    modify operation on AD/LDS
3. changePassword() given a valid user id of a user in AD/LDS and the new password
    changes the password for the user record associated with the
    modify operation on AD/LDS

We had this service deployed and running in our Production Cloud environment and both the read-only and modify operations were verified to work with out client Angular UI application.

About a week ago, another web application in our cloud started calling the unsuspendPassword over 800+ times/day. We believed that the SpringBoot 2.x REST API Service was encountering a load issue with it calling our AD/LDS server too frequently; however, by examining the tcpdump packets, we found that it was the LDAP SDK client code in our SpringBoot 2.x REST API Service that stopped responding to the AD/LDS server when it was expecting ACKs from the LDAP SDK client code.

Following are the different errors we have been seeing in our logs in the past several days:

LDAPException(resultCode=82 (local error), errorMessage='An error occurred while attempting to initialize the JAAS login context for GSSAPI authentication: LoginException(Receive timed out), ldapSDKVersion=4.0.11, revision=34e39aab27ea4fb92659a6888933db08099c7e41')

LDAPException(resultCode=82 (local error), errorMessage='An error occurred while attempting to initialize the JAAS login context for GSSAPI authentication:  LoginException(ICMP Port Unreachable), ldapSDKVersion=4.0.11, revision=34e39aab27ea4fb92659a6888933db08099c7e41')

LDAPException(resultCode=82 (local error), errorMessage='An error occurred while attempting to initialize the JAAS login context for GSSAPI authentication:  LoginException(Connection refused (Connection refused)), ldapSDKVersion=4.0.11, revision=34e39aab27ea4fb92659a6888933db08099c7e41')

After further debugging of our REST API Service (that uses the LDAP SDK), we decided to try using LDAPConnectionPool for our connections to the AD/LDS server and also upgraded to the current 5.0.1 release. This made calls to the AD/LDS server noticeably faster and worked ok with the read-only operations, but would still fail on the modify operations when the connection is closed after each modify operation. The error that were seeing for modify operation was:

"00002077: SvcErr: DSID-0338073B, problem 5012 (DIR_ERROR), data 8237\n\u0000" }, "ldap-sdk-version":"5.0.1", "ldap-sdk-revision":"3290ee33d4aa17df1aadb4d814d6534375f395a9"

But were able proceed after making code change to close the connections lasily.

After reading the online docs at https://docs.ldap.com/ldap-sdk/docs/getting-started/connections.html. We noticed that there was caution about using connection pool with StartTLS Extended operation (which we are using). So, we decided to seperate connections for Read and modify operations so that we use connection pool for read operation and non-pooled connection for modify. With this, we were able to stress test in our non-prod environments and has been holding up well, but installing to production may be problematic as the number of calls reaching is significantly higher than our non-prod environments. In prod, it still throws each of the three errors I have listed above but not any particular pattern.

dirmgr commented 4 years ago

First, let me say that I’m very much not an Active Directory expert. However, I hope I may be able to provide some assistance in this case.

The original exception messages (receive timed out, ICMP port unreachable, and connection refused) suggest that there might be a networking issue between the LDAP client and the Kerberos KDC, at least from the LDAP client’s perspective. While the “An error occurred while attempting to initialize the JAAS login context for GSSAPI authentication” portion of the message is coming from the LDAP SDK, the rest of the message is coming either from the JVM’s support for GSSAPI or perhaps the operating system’s underlying network stack.

With regard to the connection pooling issue and connections getting closed as a result of modify operations, it looks like that might be because your connection is not protected in a way that the AD server requires. For some reason, Microsoft loves providing very obscure and unhelpful messages, but according to https://docs.microsoft.com/en-us/windows/win32/debug/system-error-codes--8200-8999-, the code 8237 referenced in that message is “ERROR_DS_CONFIDENTIALITY_REQUIRED”, which should correspond to the confidentialityRequired LDAP result code (13), which indicates that the server will only permit the requested operation over a secure connection.

Since you indicate that you’re trying to use StartTLS in conjunction with a connection pool, then you should look at the StartTLSPostConnectProcessor class (https://docs.ldap.com/ldap-sdk/docs/javadoc/index.html?com/unboundid/ldap/sdk/StartTLSPostConnectProcessor.html) if you’re not already using it. A post-connect processor is intended to perform some processing after establishing a new connection for use in a connection pool, and the StartTLS post-connect processor is used to invoke the StartTLS extended operation to secure the connection. With that in place, your connections should be secure, and AD should permit operations that require secure communication.

The other possibility, since you’re using GSSAPI, would be to use the confidentiality support that it offers, and in particular the auth-conf quality of protection (QoP). It may be as simple as calling GSSAPIBindRequestProperties.setAllowedQoP with a list that only contains SASLQualityOfProtection.AUTH_CONF. Using this will encrypt the communication with a key that is established through the GSSAPI negotiation rather than one obtained through TLS negotiation.