search

BroadSoft-Xtended / BroadWorks-Dashboards-and-Discovery

This repository contains the BroadWorks Dashboards and Discovery components to extend BroadWorks data mining, reporting, and analysis capabilities.
23 stars 4 forks source link

bwlogreceiver 1204 ES user/password without truststore? #96

Closed kdotson3263k closed 4 years ago

kdotson3263k commented 4 years ago

Hi,

Is it possible to start up and utilize bwlogreceiver only using user/password without the truststore?

Also, note the "/es" extension in the "elasticserver:xxxxxxxx/es/" is this ok without and quotes?

i.e. elasticserver:xxxxx/es/

NOTE: int the props file, truststorepath:NA

2020-01-21_16:15:52.184 [main] INFO  c.b.eslogzipshared.ProcessMonitor - ProcessMonitor - MaxThreads set to: -1
2020-01-21_16:15:52.188 [main] INFO  c.b.eslogzipshared.ProcessMonitor - ProcessMonitor - MaxFiles set to: -1
2020-01-21_16:15:52.188 [main] INFO  c.b.eslogzipshared.ProcessMonitor - ProcessMonitor - MaxFilesPercentage set to: -1
2020-01-21_16:15:52.189 [main] INFO  com.broadsoft.eslogzipreceiver.Main - MAIN: Loading application properties
2020-01-21_16:15:52.191 [main] INFO  c.b.eslogzipreceiver.AppProperties - AppProperties: /bw/bwlogreceiver/bwlogreceiver_1204/logreceiver.props
2020-01-21_16:15:52.191 [main] INFO  c.b.eslogzipreceiver.AppProperties - Properties: elasticserver -> adv-elk.eaas.comcast.net/es/
2020-01-21_16:15:52.191 [main] INFO  c.b.eslogzipreceiver.AppProperties - Properties: elasticport -> 443
2020-01-21_16:15:52.191 [main] INFO  c.b.eslogzipreceiver.AppProperties - Properties: elasticclustername -> adv
2020-01-21_16:15:52.191 [main] INFO  c.b.eslogzipreceiver.AppProperties - Properties: receiverport -> 9072
2020-01-21_16:15:52.192 [main] INFO  c.b.eslogzipreceiver.AppProperties - Properties: logprocessorqueuesize -> 200
2020-01-21_16:15:52.192 [main] INFO  c.b.eslogzipreceiver.AppProperties - Properties: logprocessornumthreads -> 8
2020-01-21_16:15:52.192 [main] INFO  c.b.eslogzipreceiver.AppProperties - Properties: truststorepath -> NA
2020-01-21_16:15:52.192 [main] INFO  c.b.eslogzipreceiver.AppProperties - Properties: truststorepassword -> <hidden>
2020-01-21_16:15:52.192 [main] INFO  c.b.eslogzipreceiver.AppProperties - Properties: usekafka -> false
2020-01-21_16:15:52.192 [main] INFO  com.broadsoft.eslogzipreceiver.Main - MAIN: Initializing ElasticSearch indexer
2020-01-21_16:15:52.197 [main] INFO  c.b.e.ElasticLogIndexerThreadownerImpl - using ElasticLogIndexerThreadownerImpl
2020-01-21_16:15:52.199 [Thread-3] INFO  c.b.e.ElasticLogIndexerThreadownerImpl - starting ES indexer thread
2020-01-21_16:15:52.327 [main] INFO  c.b.e.ElasticLogIndexer - Exception creating the RestClient
java.nio.file.NoSuchFileException: NA
        at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
        at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
        at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
        at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214)
        at java.nio.file.Files.newByteChannel(Files.java:361)
        at java.nio.file.Files.newByteChannel(Files.java:407)
        at java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:384)
        at java.nio.file.Files.newInputStream(Files.java:152)
        at com.broadsoft.eslogzipreceiver.ElasticLogIndexer.startConnection(ElasticLogIndexer.java:130)
        at com.broadsoft.eslogzipreceiver.ElasticLogIndexerThreadownerImpl.startConnection(ElasticLogIndexerThreadownerImpl.java:95)
        at com.broadsoft.eslogzipreceiver.ElasticLogIndexerThreadownerImpl.<init>(ElasticLogIndexerThreadownerImpl.java:83)
        at com.broadsoft.eslogzipreceiver.ElasticLogIndexerThreadownerImpl.init(ElasticLogIndexerThreadownerImpl.java:90)
        at com.broadsoft.eslogzipreceiver.ElasticLogIndexer.init(ElasticLogIndexer.java:76)
        at com.broadsoft.eslogzipreceiver.Main.main(Main.java:60)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader.main(JarRsrcLoader.java:58)
2020-01-21_16:15:52.329 [Thread-2] INFO  c.b.eslogzipreceiver.ShutdownHandler - Caught shutdown.... Exiting nicely (Well, not yet - but to be implemented)

output from curl command:

curl -sS --insecure -u log_shipper -XGET 'https://xxxxxxx/es/_cat/indices?pretty'
Enter host password for user 'log_shipper':
green open bwlog-apache-000001      icHIMWb_STqEybmwhkaQHw 6 1  0 0   3.3kb  1.6kb
green open .apm-agent-configuration p5RzQkaeShqnl4Xf_FEe_A 1 1  0 0    566b   283b
green open bwlog-000018             Iz3GkVuVS_e4J2OWwloq9A 6 1  0 0   3.3kb  1.6kb
green open bwlog-000007             eseA6RosSXuIXmzI-7BOew 6 1  0 0   3.3kb  1.6kb
green open bwlog-000006             iq1XO97lS3-fHQnSw7um5A 6 1  0 0   3.3kb  1.6kb
green open bwlog-000017             LXZlYs8YRruqWxThPO5xcw 6 1  0 0   3.3kb  1.6kb
green open bwlog-000009             iAwl3X4dRWyQhLnbbrTb5Q 6 1  0 0   3.3kb  1.6kb
green open bwlog-000008             CWewV7C8RNy4Ml2E9NfhxQ 6 1  0 0   3.3kb  1.6kb
green open bwlog-000003             0WaIqnyQS6O4eb-0IaqbHg 6 1  0 0   3.3kb  1.6kb
green open bwlog-000014             WYZNywSBQtGblZs40JJ2yw 6 1  0 0   3.3kb  1.6kb
green open bwlog-000013             JtZMIrwpQS6MOlI39S0stA 6 1  0 0   3.3kb  1.6kb
green open .kibana_1                OjyP0nzaRI26fn1qlv7_rQ 1 1 15 5  85.5kb 42.7kb
green open bwlog-000002             JPD53n1TQmydMajybHt4zQ 6 1  0 0   3.3kb  1.6kb
green open bwlog-000016             WAzCEV0SQYGQrs-ICoYWmg 6 1  0 0   3.3kb  1.6kb
green open bwlog-000005             pjXJdKrJRO2r9APKYUgYfA 6 1  0 0   3.3kb  1.6kb
green open bwlog-000004             gbj5DdWpS6qZ0W1fEFhejQ 6 1  0 0   3.3kb  1.6kb
green open bwlog-000015             nCQgE6cVSP2DKX46MwVXhg 6 1  0 0   3.3kb  1.6kb
green open bwlog-000010             iIqWnhflSfCC1pijp9WPYQ 6 1  0 0   3.3kb  1.6kb
green open .security-7              W601OjB_TVW6Mt1Qjm5SyA 1 1 33 6 131.6kb 65.8kb
green open bwlog-000001             cGNVGc3DSLOv2jNgRrOGPg 6 1  0 0   3.3kb  1.6kb
green open bwlog-000012             fmO5bgtrSi68KPSxG_Q0Aw 6 1  0 0   3.3kb  1.6kb
green open bwlog-000011             8tuWq8FyT_2jgNyNgBNPzg 6 1  0 0   3.3kb  1.6kb
green open .kibana_task_manager_1   SdhntzWNRJKoznrjS8r7gw 1 1  2 1  53.7kb 26.8kb
gokulbsft commented 4 years ago

Currently, it is not supported without truststore file.

vivekgplus commented 4 years ago

in props file

senderreceiverusessl:false

make a empty file yourkeystore.jks

truststorepath:yourkeystore.jks truststorepassword:123456

This should work

kdotson3263k commented 4 years ago

I attempted vivekgplus suggestion, with and without the senderreceiverusessl. Baiscally, I created a truststore then deleted the alias so that the trustore is empty. See logs below.

I am working on getting a cert implemented onto our elasticsearch cluster, but there are currently other users not ready for the implementation so it could be some time.

2020-01-22_15:45:29.457 [main] INFO  c.b.eslogzipshared.ProcessMonitor - ProcessMonitor - MaxThreads set to: -1
2020-01-22_15:45:29.460 [main] INFO  c.b.eslogzipshared.ProcessMonitor - ProcessMonitor - MaxFiles set to: -1
2020-01-22_15:45:29.460 [main] INFO  c.b.eslogzipshared.ProcessMonitor - ProcessMonitor - MaxFilesPercentage set to: -1
2020-01-22_15:45:29.462 [main] INFO  com.broadsoft.eslogzipreceiver.Main - MAIN: Loading application properties
2020-01-22_15:45:29.464 [main] INFO  c.b.eslogzipreceiver.AppProperties - AppProperties: /bw/bwlogreceiver/bwlogreceiver_1204/logreceiver.props
2020-01-22_15:45:29.464 [main] INFO  c.b.eslogzipreceiver.AppProperties - Properties: elasticserver -> xx.xx.xx.xx/es/
2020-01-22_15:45:29.464 [main] INFO  c.b.eslogzipreceiver.AppProperties - Properties: elasticport -> 443
2020-01-22_15:45:29.464 [main] INFO  c.b.eslogzipreceiver.AppProperties - Properties: elasticclustername -> adv
2020-01-22_15:45:29.465 [main] INFO  c.b.eslogzipreceiver.AppProperties - Properties: receiverport -> 9072
2020-01-22_15:45:29.465 [main] INFO  c.b.eslogzipreceiver.AppProperties - Properties: logprocessorqueuesize -> 200
2020-01-22_15:45:29.465 [main] INFO  c.b.eslogzipreceiver.AppProperties - Properties: logprocessornumthreads -> 8
2020-01-22_15:45:29.465 [main] INFO  c.b.eslogzipreceiver.AppProperties - Properties: truststorepath -> /bw/bwlogreceiver/bwlogreceiver/keystore.jks
2020-01-22_15:45:29.465 [main] INFO  c.b.eslogzipreceiver.AppProperties - Properties: truststorepassword -> <hidden>
2020-01-22_15:45:29.465 [main] INFO  c.b.eslogzipreceiver.AppProperties - Properties: usekafka -> false
2020-01-22_15:45:29.465 [main] INFO  com.broadsoft.eslogzipreceiver.Main - MAIN: Initializing ElasticSearch indexer
2020-01-22_15:45:29.469 [main] INFO  c.b.e.ElasticLogIndexerThreadownerImpl - using ElasticLogIndexerThreadownerImpl
2020-01-22_15:45:29.472 [Thread-3] INFO  c.b.e.ElasticLogIndexerThreadownerImpl - starting ES indexer thread
2020-01-22_15:45:29.975 [main] INFO  c.b.e.ElasticLogIndexer - Exception creating the RestClient
java.net.UnknownHostException: xx.xx.xx.xx/es/: Name or service not known
        at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method)
        at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:929)
        at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1324)
        at java.net.InetAddress.getAllByName0(InetAddress.java:1277)
        at java.net.InetAddress.getAllByName(InetAddress.java:1193)
        at java.net.InetAddress.getAllByName(InetAddress.java:1127)
        at org.apache.http.impl.conn.SystemDefaultDnsResolver.resolve(SystemDefaultDnsResolver.java:45)
        at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager$InternalAddressResolver.resolveRemoteAddress(PoolingNHttpClientConnectionManager.java:609)
        at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager$InternalAddressResolver.resolveRemoteAddress(PoolingNHttpClientConnectionManager.java:580)
        at org.apache.http.nio.pool.AbstractNIOConnPool.processPendingRequest(AbstractNIOConnPool.java:427)
        at org.apache.http.nio.pool.AbstractNIOConnPool.lease(AbstractNIOConnPool.java:276)
        at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.requestConnection(PoolingNHttpClientConnectionManager.java:266)
        at org.apache.http.impl.nio.client.AbstractClientExchangeHandler.requestConnection(AbstractClientExchangeHandler.java:363)
        at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.start(DefaultClientExchangeHandlerImpl.java:125)
        at org.apache.http.impl.nio.client.InternalHttpAsyncClient.execute(InternalHttpAsyncClient.java:141)
        at org.apache.http.impl.nio.client.CloseableHttpAsyncClient.execute(CloseableHttpAsyncClient.java:68)
        at org.elasticsearch.client.RestClient.performRequestAsync(RestClient.java:300)
        at org.elasticsearch.client.RestClient.performRequestAsync(RestClient.java:290)
        at org.elasticsearch.client.RestClient.performRequest(RestClient.java:211)
        at org.elasticsearch.client.RestClient.performRequest(RestClient.java:184)
        at org.elasticsearch.client.RestClient.performRequest(RestClient.java:163)
        at com.broadsoft.eslogzipreceiver.ElasticLogIndexer.startConnection(ElasticLogIndexer.java:149)
        at com.broadsoft.eslogzipreceiver.ElasticLogIndexerThreadownerImpl.startConnection(ElasticLogIndexerThreadownerImpl.java:95)
        at com.broadsoft.eslogzipreceiver.ElasticLogIndexerThreadownerImpl.<init>(ElasticLogIndexerThreadownerImpl.java:83)
        at com.broadsoft.eslogzipreceiver.ElasticLogIndexerThreadownerImpl.init(ElasticLogIndexerThreadownerImpl.java:90)
        at com.broadsoft.eslogzipreceiver.ElasticLogIndexer.init(ElasticLogIndexer.java:76)
        at com.broadsoft.eslogzipreceiver.Main.main(Main.java:60)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader.main(JarRsrcLoader.java:58)
2020-01-22_15:45:29.977 [Thread-2] INFO  c.b.eslogzipreceiver.ShutdownHandler - Caught shutdown.... Exiting nicely (Well, not yet - but to be implemented)
gokulbsft commented 4 years ago

bwlogreceiver can not connect to the elastic server with a custom context path ("es" - in your case).

kdotson3263k commented 4 years ago

Hello, Update on this issue, our data team did provide a "non-custom" context path, but we are still missing certs for a keystore. As I said above, I attempted to work around by adding a empty keystore, but I am receiving the following error.

2020-02-18_15:32:36.088 [main] INFO  com.broadsoft.eslogzipreceiver.Main - MAIN: Initializing ElasticSearch indexer
2020-02-18_15:32:36.092 [main] INFO  c.b.e.ElasticLogIndexerThreadownerImpl - using ElasticLogIndexerThreadownerImpl
2020-02-18_15:32:36.095 [Thread-3] INFO  c.b.e.ElasticLogIndexerThreadownerImpl - starting ES indexer thread
2020-02-18_15:32:36.745 [main] INFO  c.b.e.ElasticLogIndexer - Exception creating the RestClient
javax.net.ssl.SSLException: java.lang.RuntimeException: Unexpected error: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty
        at org.apache.http.nio.reactor.ssl.SSLIOSession.convert(SSLIOSession.java:260)
        at org.apache.http.nio.reactor.ssl.SSLIOSession.doUnwrap(SSLIOSession.java:275)
        at org.apache.http.nio.reactor.ssl.SSLIOSession.doHandshake(SSLIOSession.java:328)
        at org.apache.http.nio.reactor.ssl.SSLIOSession.isAppInputReady(SSLIOSession.java:509)
        at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:120)
        at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162)
        at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337)
        at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315)
        at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276)
        at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
        at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:588)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: Unexpected error: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty
        at sun.security.validator.PKIXValidator.<init>(PKIXValidator.java:91)
        at sun.security.validator.Validator.getInstance(Validator.java:181)
        at sun.security.ssl.X509TrustManagerImpl.getValidator(X509TrustManagerImpl.java:312)
        at sun.security.ssl.X509TrustManagerImpl.checkTrustedInit(X509TrustManagerImpl.java:171)
        at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:239)
        at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:136)
        at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1626)
        at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:223)
        at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1037)
        at sun.security.ssl.Handshaker$1.run(Handshaker.java:970)
        at sun.security.ssl.Handshaker$1.run(Handshaker.java:967)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1459)
        at org.apache.http.nio.reactor.ssl.SSLIOSession.doRunTask(SSLIOSession.java:283)
        at org.apache.http.nio.reactor.ssl.SSLIOSession.doHandshake(SSLIOSession.java:353)
        ... 9 common frames omitted
Caused by: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty
        at java.security.cert.PKIXParameters.setTrustAnchors(PKIXParameters.java:200)
        at java.security.cert.PKIXParameters.<init>(PKIXParameters.java:120)
        at java.security.cert.PKIXBuilderParameters.<init>(PKIXBuilderParameters.java:104)
        at sun.security.validator.PKIXValidator.<init>(PKIXValidator.java:89)
        ... 23 common frames omitted
2020-02-18_15:32:36.749 [Thread-2] INFO  c.b.eslogzipreceiver.ShutdownHandler - Caught shutdown.... Exiting nicely (Well, not yet - but to be implemented)