search

prestodb / presto

The official home of the Presto distributed SQL query engine for big data
http://prestodb.io
Apache License 2.0
15.91k stars 5.33k forks source link

LDAP authentication #8569

Closed moshir closed 2 years ago

moshir commented 7 years ago

Hi I setup LDAP integration as described in Presto & Terrada documentation with a local openldap server on the same host as presto coordinator. I keep on getting the same error :

[hadoop@ip-123-45-67-890 ~]$java -Djavax.net.debug=ssl -jar $PRESTO_HOME/bin/presto-cli-0.170-executable --server https://prestodb.issueexample.com.com:8443  --user user1 --password  --catalog hive --schema default --keystore-path /home/hadoop/certificates/myserver.keystore --keystore-password <passwd>

http-client-anonymous1-26, fatal error: 46: General SSLEngine problem
java.security.cert.CertificateException: No name matching prestodb.issueexample.com found `

Java ssl log shows my certificates are found :

***
adding as trusted cert:
  Subject: CN=<hostname>, OU=PositiveSSL, OU=Domain Control Validated
  Issuer:  CN=COMODO RSA Domain Validation Secure Server CA, O=COMODO CA Limited, L=Salford, ST=Greater Manchester, C=GB
  Algorithm: RSA; Serial number: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
  Valid from Fri Jul 21 00:00:00 UTC 2017 until Sat Jul 21 23:59:59 UTC 2018

Certificates I'm using are found in the keystore :

[hadoop@ip-123-45-67-890 ~]$ keytool -list -alias prestodb.issueexample.com -keystore /home/hadoop/certificates/myserver.keystore
Enter keystore password:
prestodb.issueexample.com, Jul 21, 2017, trustedCertEntry,
Certificate fingerprint (SHA1): xx:xx:xx:xx:xxx...

Openldap is configure locally with the same certificates as my presto keystore: slapd.conf

TLSCACertificatePath /home/hadoop/certificates/cacertificate.cer
TLSCertificateFile /home/hadoop/certificates/certificate.pem
TLSCertificateKeyFile /home/hadoop/certificates/certificateKey.pem

cn\=config.ldif

olcTLSCACertificatePath: /home/hadoop/certificates/cacertificate.pem
olcTLSCertificateFile: /home/hadoop/certificates/certificate.pem
olcTLSCertificateKeyFile: /home/hadoop/certificates/certificateKey.pem

I dont have kerberos setup. Do i need it ? There's a line in the ldap documentation talknig about a principal :

the alias in the keytool command line should match the principal that the Presto coordinator will use.

Apart from that, I'm not sure what i'm doing wrong and would really appreciate some guidance to debug. Also, is there anyway to have basic user/password working with Presto, even without LDAP ?

Thank you !

moshir commented 7 years ago

After some more trials, i get:

Error running command: Server refused connection: ...

in server.log : 

Caused by: javax.naming.CommunicationException: anonymous bind failed: ldaphost

Does anyone have any clue about what's configured wrong ?

akshatnair commented 7 years ago

Can you share your configuration file? Also, there is a troubleshooting section in the documentation, give it a try if you haven't already.

moshir commented 7 years ago

Certificates

  1. I created self-signed certificates using opensslperl : CA.pl
  2. ALL certificates were checked with openssl and all certificates are consistent
  3. I imported CA certifiates, server certificate and key in openldap running on the same host as the master node/presto coordinator and configured with LDAPS

Server certificate hosting presto and ldap are configured with FQDN pasted from $ hostname which returns

ip-172-xx-xx-1

On the coordinator

  1. CA certificate was imported to java cacert : sudo keytool -import -keystore /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.131-2.b11.30.amzn1.x86_64/jre/lib/security/cacerts -trustcacerts -alias ldap_server -file cacert.pem
  2. a keystore was generated for presto : keytool -genkeypair -alias presto -keyalg RSA -keystore keystore.jks
  3. presto default configuration was configured as follows : 
    
http-server.authentication.type=LDAP
authentication.ldap.url=ldaps://ip-172-xx-xx-1:389
authentication.ldap.user-bind-pattern=uid=tomas,OU=dev,DC=example,DC=com

http-server.https.enabled=true http-server.https.port=8443 http-server.https.keystore.path=/home/hadoop/cert/keystore.jks http-server.https.keystore.key=arandompassword


openldap configured with SSL but listening on port 389

Starting presto-cli with following options , i get :

[hadoop@ip-172-xx-xx-x etc]$ presto-cli --server https://ip-172-xx-xx-x:8443 --user tomas --password --catalog hive Password: presto> use hive.default ; presto:default> select * from A ; Error running command: javax.net.ssl.SSLHandshakeException: General SSLEngine problem


in server.log :

2017-07-25T12:43:08.609Z ERROR main com.facebook.presto.server.PrestoServer Unable to create injector, see the following errors:

1) Error injecting constructor, java.lang.RuntimeException: javax.naming.CommunicationException: anonymous bind failed: ip-172-xx-xx-1:389 [Root exception is javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake] at com.facebook.presto.server.security.LdapFilter.(LdapFilter.java:92) while locating com.facebook.presto.server.security.LdapFilter at com.facebook.presto.server.security.ServerSecurityModule.lambda$setup$3(ServerSecurityModule.java:53) (via modules: com.facebook.presto.server.security.ServerSecurityModule -> io.airlift.configuration.ConditionalModule -> com.facebook.presto.server.security.ServerSecurityModule$$Lambda$27/366252104) while locating javax.servlet.Filter annotated with @com.google.inject.multibindings.Element(setName=@io.airlift.http.server.TheServlet,uniqueId=339, type=MULTIBINDER, keyType=) at io.airlift.http.server.HttpServerModule.configure(HttpServerModule.java:68) (via modules: io.airlift.http.server.HttpServerModule -> com.google.inject.multibindings.Multibinder$RealMultibinder) while locating java.util.Set annotated with @io.airlift.http.server.TheServlet() for parameter 4 at io.airlift.http.server.HttpServerProvider.(HttpServerProvider.java:72)

anusudarsan commented 7 years ago

@moshir Couple of things. 1) Looks like the Presto server did not start successfully. There seems to be connectivity issues from the Presto coordinator to your LDAP server. Are you sure you have LDAPS enabled on port 389? By default, the LDAPS is enabled on port 636. Did you explicitly change it to be on 389? 2) You could verify the connectivity between LDAP server and the coordinator node by using a utility called ldapsearch . See this thread for example usage - https://groups.google.com/forum/#!topic/presto-users/-h3m5QW-C_M 3) Once you have Presto server up and running, you should be able to use the cli. But make sure you pass the SSL related parameters too (--keystore-path & --keystore-password). See documentation here - https://prestodb.io/docs/current/security/ldap.html#presto-cli-execution

liusztc09 commented 7 years ago

Hi @anusudarsan Our team faced the similar issue. It looks like the current implementation doesn't make an authorized with the bind user and pswd, instead it makes anonymous bind to the AD. When the AD allows anonymous bind it works but in many prod env anonymous bind is not enabled. Any suggestion to that or can we easily add support for setting bind username and pswd? Thanks!

xor007 commented 6 years ago

same here: version 1.67-t seems to try anonymous bind and hits Caused by: java.lang.RuntimeException: javax.naming.AuthenticationException: [LDAP: error code 49 - INVALID_CREDENTIALS: Bind failed: null]

BourneYu commented 5 years ago

Hi @anusudarsan Our team faced the similar issue. It looks like the current implementation doesn't make an authorized with the bind user and pswd, instead it makes anonymous bind to the AD. When the AD allows anonymous bind it works but in many prod env anonymous bind is not enabled. Any suggestion to that or can we easily add support for setting bind username and pswd? Thanks!

Hi, have you implemented presto security certification through AD? I also encountered a similar problem.

BourneYu commented 5 years ago

Hi @anusudarsan Our team faced the similar issue. It looks like the current implementation doesn't make an authorized with the bind user and pswd, instead it makes anonymous bind to the AD. When the AD allows anonymous bind it works but in many prod env anonymous bind is not enabled. Any suggestion to that or can we easily add support for setting bind username and pswd? Thanks!

@liusztc09 Hi, have you implemented presto security certification through AD? I also encountered a similar problem.

kbajda commented 5 years ago

@xor007 @BourneYu This issue is already fixed in the latest release of Presto from Starburst (a free download from www.starburstdata.com). Please let me know if that helped in your setup.

BourneYu commented 5 years ago

Hi @kbajda @anusudarsan ,I install Presto(v0.213) and setup AD(LDAPS) integration as described in https://prestodb.io/docs/current/security/ldap.html#presto-cli-execution . Java Version: java-1.8.0-openjdk-1.8.0.191.b12

bin/launcher run --verbose,I getting the below error : Error injecting constructor, java.lang.RuntimeException: javax.naming.CommunicationException: anonymous bind failed: ds.dev.opg.cn:636 [Root exception is javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateException: No subject alternative DNS name matching ds.dev.opg.cn found.

The steps are as follows:

configuration files are as follows:

etc/config.properties: coordinator=true node-scheduler.include-coordinator=false http-server.http.port=8080 discovery-server.enabled=true discovery.uri=http://10.205.92.157:8080 query.max-memory=1PB query.max-memory-per-node=2994MB query.max-total-memory-per-node=2994MB

http-server.authentication.type=PASSWORD http-server.https.enabled=true http-server.https.port=8843 http-server.https.keystore.path=/etc/presto/keystore.jks http-server.https.keystore.key=changeit

etc/password-authenticator.properties : password-authenticator.name=ldap ldap.url=ldaps://ds.dev.opg.cn:636 ldap.user-bind-pattern=${USER}@ds.dev.opg.cn ldap.user-base-dn=DC=ds,DC=dev,DC=opg,DC=cn

Following the advice (https://github.com/prestodb/presto/pull/11714/files/77aa30df30bce996a23d07f511d3de0ca4df5822), I modified the code and redeployed it. Presto started successfully, but when logging in via https：https://10.205.92.157:8843/ui/, I reported the following error:

java.lang.RuntimeException: Authentication error at com.facebook.presto.server.security.PasswordAuthenticator.authenticate(PasswordAuthenticator.java:71) at com.facebook.presto.server.security.AuthenticationFilter.doFilter(AuthenticationFilter.java:80) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at io.airlift.http.server.TraceTokenFilter.doFilter(TraceTokenFilter.java:64) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at io.airlift.http.server.TimingFilter.doFilter(TimingFilter.java:52) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:740) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1340) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1242) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126) at org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:174) at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:61) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.Server.handle(Server.java:503) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:411) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:305) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:159) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683) at java.lang.Thread.run(Thread.java:748) Caused by: com.google.common.util.concurrent.UncheckedExecutionException: java.lang.RuntimeException: Authentication error: user:test password:test001 at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2052) at com.google.common.cache.LocalCache.get(LocalCache.java:3943) at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3967) at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4952) at com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4958) at com.facebook.presto.password.LdapAuthenticator.createAuthenticatedPrincipal(LdapAuthenticator.java:90) at com.facebook.presto.server.security.PasswordAuthenticator.authenticate(PasswordAuthenticator.java:65) ... 39 more

Thanks in advance!!!

dain commented 5 years ago

I don't know much about LDAP, but the error looks like it is coming from basic SSL setup and not LDAP in particular. This error SSLHandshakeException: CertificateException: No subject alternative DNS name matching ds.dev.opg.cn found" means that you are attempting to connect to a host namedds.dev.opg.cn` (which was specified in your config), and the host is returning a certificate that does not contain that host name. Therefore, client doesn't trust that host and won't finish the connection establishment. You should be able to dump the actual server certificate with OpenSSL (something like https://stackoverflow.com/a/7886248/2958751) and verify the certificate is incorrect.

findepi commented 5 years ago

This will be fixed in Presto 301 (https://github.com/prestosql/presto/pull/97)

HarvinderBhullar commented 4 years ago

I was facing similar issue, It worked for me when I added CERTIFICATE after password (Check your certificate as mentioned above before trying this) http-server.authentication.type=PASSWORD, CERTIFICATE

stale[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had any activity in the last 2 years. If you feel that this issue is important, just comment and the stale tag will be removed; otherwise it will be closed in 7 days. This is an attempt to ensure that our open issues remain valuable and relevant so that we can keep track of what needs to be done and prioritize the right things.