marklogic / ml-gradle

Gradle plugin for automating everything involving MarkLogic
Other
73 stars 77 forks source link

ml-gradle basic authentication with SSL exceptions #304

Closed tomaszs86 closed 6 years ago

tomaszs86 commented 6 years ago

When calling mlDeploy using ml-gradle application throws exception during instalation custom web services located in ml-modules/services folder.

ml-gradle 3.0.0 exception message:

Exception in thread "ThreadPoolTaskExecutor-5" com.sun.jersey.api.client.ClientHandlerException: java.net.SocketException: Connection reset
         at com.sun.jersey.client.apache4.ApacheHttpClient4Handler.handle(ApacheHttpClient4Handler.java:184)
         at com.marklogic.client.impl.DigestChallengeFilter.handle(DigestChallengeFilter.java:34)
         at com.sun.jersey.api.client.filter.HTTPDigestAuthFilter.handle(HTTPDigestAuthFilter.java:493)
         at com.sun.jersey.api.client.Client.handle(Client.java:648)
         at com.sun.jersey.api.client.WebResource.head(WebResource.java:176)
         at com.marklogic.client.impl.JerseyServices.makeFirstRequest(JerseyServices.java:505)
         at com.marklogic.client.impl.JerseyServices.putPostValueImpl(JerseyServices.java:2748)
         at com.marklogic.client.impl.JerseyServices.putValue(JerseyServices.java:2693)
         at com.marklogic.client.impl.ResourceExtensionsImpl.writeServices(ResourceExtensionsImpl.java:206)
         at com.marklogic.client.ext.modulesloader.impl.DefaultModulesLoader$1.run(DefaultModulesLoader.java:428)
         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
         at java.lang.Thread.run(Thread.java:748)

 Exception in thread "ThreadPoolTaskExecutor-5" com.sun.jersey.api.client.ClientHandlerException: java.net.SocketException: Connection reset
    at com.sun.jersey.client.apache4.ApacheHttpClient4Handler.handle(ApacheHttpClient4Handler.java:184)
    at com.marklogic.client.impl.DigestChallengeFilter.handle(DigestChallengeFilter.java:34)
    at com.sun.jersey.api.client.filter.HTTPDigestAuthFilter.handle(HTTPDigestAuthFilter.java:493)
    at com.sun.jersey.api.client.Client.handle(Client.java:648)
    at com.sun.jersey.api.client.WebResource.head(WebResource.java:176)
    at com.marklogic.client.impl.JerseyServices.makeFirstRequest(JerseyServices.java:505)
    at com.marklogic.client.impl.JerseyServices.putPostValueImpl(JerseyServices.java:2748)
    at com.marklogic.client.impl.JerseyServices.putValue(JerseyServices.java:2693)
    at com.marklogic.client.impl.ResourceExtensionsImpl.writeServices(ResourceExtensionsImpl.java:206)
    at com.marklogic.client.ext.modulesloader.impl.DefaultModulesLoader$1.run(DefaultModulesLoader.java:428)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

ml-gradle 3.4.0 exception message:

 Error caught while loading modules, cause: java.net.SocketException: Connection reset
 com.marklogic.client.MarkLogicIOException: java.net.SocketException: Connection reset
         at com.marklogic.client.impl.OkHttpServices.sendRequestOnce(OkHttpServices.java:697)
         at com.marklogic.client.impl.OkHttpServices.putPostValueImpl(OkHttpServices.java:2810)
         at com.marklogic.client.impl.OkHttpServices.putValue(OkHttpServices.java:2740)
         at com.marklogic.client.impl.ResourceExtensionsImpl.writeServices(ResourceExtensionsImpl.java:206)
         at com.marklogic.client.ext.modulesloader.impl.DefaultModulesLoader.lambda$installService$1(DefaultModulesLoader.java:408)
         at com.marklogic.client.ext.modulesloader.impl.DefaultModulesLoader.lambda$executeTask$5(DefaultModulesLoader.java:501)
         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
         at java.lang.Thread.run(Thread.java:748)

 Error caught while loading modules, cause: java.net.ProtocolException: Unexpected status line: !DOCTYPE HTML PUBLIC '-//IETF//DTD HTML 2.0//EN'>
 com.marklogic.client.MarkLogicIOException: java.net.ProtocolException: Unexpected status line: <!DOCTYPE HTML PUBLIC '-//IETF//DTD HTML 2.0//EN'>
         at com.marklogic.client.impl.OkHttpServices.sendRequestOnce(OkHttpServices.java:697)
         at com.marklogic.client.impl.OkHttpServices.putPostValueImpl(OkHttpServices.java:2810)
         at com.marklogic.client.impl.OkHttpServices.putValue(OkHttpServices.java:2740)
         at com.marklogic.client.impl.ResourceExtensionsImpl.writeServices(ResourceExtensionsImpl.java:206)
         at com.marklogic.client.ext.modulesloader.impl.DefaultModulesLoader.lambda$installService$1(DefaultModulesLoader.java:408)
         at com.marklogic.client.ext.modulesloader.impl.DefaultModulesLoader.lambda$executeTask$5(DefaultModulesLoader.java:501)
         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
         at java.lang.Thread.run(Thread.java:748)

Configuration: App servers on port 8000, 8001, 8002 have basic authentication enabled, internal security enabled and SSL enabled. My custom app server on port 8020 has this same configuration.

gradle.properties:

mlAppServicesAuthentication=basic
mlAdminAuthentication=basic
mlManageAuthentication=basic

mlAdminScheme=https
mlManageScheme=https
mlManageSimpleSsl=true
mlAdminSimpleSsl=true

SSL default options enabled in custom-app-server.json

"authentication" : "basic"
"ssl-allow-sslv3" : true
"ssl-allow-tls" : true
"ssl-disable-sslv3" : false
"ssl-disable-tlsv1" : false
"ssl-disable-tlsv1-1" : false
"ssl-disable-tlsv1-2" : false
"ssl-ciphers" : "ALL:!LOW:@STRENGTH"
"ssl-require-client-certificate" : true
"ssl-hostname" : "host-name-here"
"ssl-certificate-template" : "certificate-name-here"

The module database is filled with data, but my custom web services are missing because of exceptions. This same bug is in DHF.

rjrudin commented 6 years ago

Hopefully this is the same fix as #303 - set mlRestAuthentication=basic, and mlRestSimpleSsl=true as well.

tomaszs86 commented 6 years ago

I enabled following setting, but it still does not work with gradle 3.0.0 and 3.4.0

mlAdminScheme=https
mlManageScheme=https

mlManageSimpleSsl=true
mlAdminSimpleSsl=true
mlRestSimpleSsl=true
mlAppServicesSimpleSsl=true

mlAppServicesAuthentication=basic
mlRestAuthentication=basic
mlAdminAuthentication=basic
mlManageAuthentication=basic

SSL default options enabled in custom-app-server.json

"authentication" : "basic"
"ssl-allow-sslv3" : true
"ssl-allow-tls" : true
"ssl-disable-sslv3" : false
"ssl-disable-tlsv1" : false
"ssl-disable-tlsv1-1" : false
"ssl-disable-tlsv1-2" : false
"ssl-ciphers" : "ALL:!LOW:@STRENGTH"
"ssl-require-client-certificate" : true
"ssl-hostname" : "host-name-here"
"ssl-certificate-template" : "certificate-name-here"

ml-gradle 3.0.0

Before mlDeploy I undeployed app. New exception regarding amps appeared. Now there are two different exceptions, one regarding custom web services and one regarding amps.

Exception in thread "ThreadPoolTaskExecutor-1" com.sun.jersey.api.client.ClientHandlerException: java.net.SocketException: Connection reset
        at com.sun.jersey.client.apache4.ApacheHttpClient4Handler.handle(ApacheHttpClient4Handler.java:184)
        at com.sun.jersey.api.client.filter.HTTPBasicAuthFilter.handle(HTTPBasicAuthFilter.java:104)
        at com.sun.jersey.api.client.Client.handle(Client.java:648)
        at com.sun.jersey.api.client.WebResource.handle(WebResource.java:680)
        at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
        at com.sun.jersey.api.client.WebResource$Builder.put(WebResource.java:537)
        at com.marklogic.client.impl.JerseyServices.putPostValueImpl(JerseyServices.java:2766)
        at com.marklogic.client.impl.JerseyServices.putValue(JerseyServices.java:2693)
        at com.marklogic.client.impl.ResourceExtensionsImpl.writeServices(ResourceExtensionsImpl.java:206)
        at com.marklogic.client.ext.modulesloader.impl.DefaultModulesLoader$1.run(DefaultModulesLoader.java:428)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
...

Executing command [com.marklogic.appdeployer.command.security.DeployAmpsCommand] with sort order [450]
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:57)
        at com.sun.jersey.client.apache4.ApacheHttpClient4Handler.handle(ApacheHttpClient4Handler.java:170)
        ... 12 more
Exception in thread "ThreadPoolTaskExecutor-4" com.sun.jersey.api.client.ClientHandlerException: java.net.SocketException: Connection reset
Processing files in directory: path-to-project\src\main\ml-config\security\amps
        at com.sun.jersey.client.apache4.ApacheHttpClient4Handler.handle(ApacheHttpClient4Handler.java:184)
        at com.sun.jersey.api.client.filter.HTTPBasicAuthFilter.handle(HTTPBasicAuthFilter.java:104)
        at com.sun.jersey.api.client.Client.handle(Client.java:648)
        at com.sun.jersey.api.client.WebResource.handle(WebResource.java:680)
        at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
        at com.sun.jersey.api.client.WebResource$Builder.put(WebResource.java:537)
        at com.marklogic.client.impl.JerseyServices.putPostValueImpl(JerseyServices.java:2766)
        at com.marklogic.client.impl.JerseyServices.putValue(JerseyServices.java:2693)
        at com.marklogic.client.impl.ResourceExtensionsImpl.writeServices(ResourceExtensionsImpl.java:206)
        at com.marklogic.client.ext.modulesloader.impl.DefaultModulesLoader$1.run(DefaultModulesLoader.java:428)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Processing file: path-to-project\src\main\ml-config\security\amps\custom-amped.json
Caused by: java.net.SocketException: Connection reset
        at java.net.SocketInputStream.read(SocketInputStream.java:210)
        at java.net.SocketInputStream.read(SocketInputStream.java:141)
        at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
        at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
        at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
        at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
        at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
        at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:260)
        at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
        at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
        at org.apache.http.impl.conn.AbstractClientConnAdapter.receiveResponseHeader(AbstractClientConnAdapter.java:223)
        at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:271)
        at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:123)
        at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
        at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
        at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:863)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:115)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:57)
        at com.sun.jersey.client.apache4.ApacheHttpClient4Handler.handle(ApacheHttpClient4Handler.java:170)
        ... 12 more
Sending XML GET request as user with admin role 'admin' to path: /manage/v2/amps

ml-gradle 3.4.0

There is only exception regarding custom web services

Error caught while loading modules, cause: java.net.ProtocolException: Unexpected status line: <!DOCTYPE HTML PUBLIC '-//IETF//DTD HTML 2.0//EN'>
com.marklogic.client.MarkLogicIOException: java.net.ProtocolException: Unexpected status line: <!DOCTYPE HTML PUBLIC '-//IETF//DTD HTML 2.0//EN'>
        at com.marklogic.client.impl.OkHttpServices.sendRequestOnce(OkHttpServices.java:697)
        at com.marklogic.client.impl.OkHttpServices.putPostValueImpl(OkHttpServices.java:2810)
        at com.marklogic.client.impl.OkHttpServices.putValue(OkHttpServices.java:2740)
        at com.marklogic.client.impl.ResourceExtensionsImpl.writeServices(ResourceExtensionsImpl.java:206)
        at com.marklogic.client.ext.modulesloader.impl.DefaultModulesLoader.lambda$installService$1(DefaultModulesLoader.java:408)
        at com.marklogic.client.ext.modulesloader.impl.DefaultModulesLoader.lambda$executeTask$5(DefaultModulesLoader.java:501)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.ProtocolException: Unexpected status line: <!DOCTYPE HTML PUBLIC '-//IETF//DTD HTML 2.0//EN'>
        at okhttp3.internal.http.StatusLine.parse(StatusLine.java:69)
        at okhttp3.internal.http1.Http1Codec.readResponseHeaders(Http1Codec.java:187)
        at okhttp3.internal.http.CallServerInterceptor.intercept(CallServerInterceptor.java:88)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
        at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:45)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
        at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
        at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
        at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:125)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
        at com.marklogic.client.impl.HTTPBasicAuthInterceptor.intercept(HTTPBasicAuthInterceptor.java:31)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
        at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:200)
        at okhttp3.RealCall.execute(RealCall.java:77)
        at com.marklogic.client.impl.OkHttpServices.sendRequestOnce(OkHttpServices.java:695)
        ... 8 more

Releasing connection
Finished executing command [com.marklogic.appdeployer.command.modules.LoadModulesCommand]
rjrudin commented 6 years ago

With 3.4.0, use mlSimpleSsl instead of mlRestSimpleSsl (I wish it were named the latter, but the former was created a long time ago, before the other "SimpleSsl" properties were created).

Also, for debugging, try the instructions at https://github.com/marklogic-community/ml-gradle/wiki/Debugging-module-loading to see if you can configure the Java Client to talk to your SSL REST server. ml-gradle isn't doing anything special under the hood, it's just configuring the Java Client to do everything.

tomaszs86 commented 6 years ago

Many thanks. Now it works. I will try debugging.