search

TheHive-Project / TheHive

TheHive: a Scalable, Open Source and Free Security Incident Response Platform
https://thehive-project.org
GNU Affero General Public License v3.0
3.42k stars 623 forks source link

TheHive 3.2.1-1 and ElasticSearch 5.6.12 X-Pack Integration #891

Open kara-1234 opened 5 years ago

kara-1234 commented 5 years ago

Request Type

Bug

Work Environment

Question Answer
OS version (server) Cent OS 7
OS version (client) Windows 10
TheHive version / git hash 3.2.1-1
ElasticSeach 5.6.12
Package Type RPM
Browser type & version FF

Problem Description

I am running TheHive 3.2.1-1 and Elastic 5.6.12 without any problems. When I enabled X-Pack get "ElasticSearch Cluster is Unavailable"

Steps to Reproduce

  1. Install TheHive 3.2.1-1 on one server
  2. Install ElasticSearch on another server with X-Pack, but keep X-Pack off
  3. Make sure everything is working
  4. Turn on X-Pack

Complementary information

TheHive Config

search.username = "XXXX"
search.password = "XXXX"
search.ssl.enabled = true
search.ssl.ca = "/opt/thehive/certs/private.pem"
search.ssl.certificate = "/opt/thehive/certs/server.pem"
search.ssl.key = "/opt/thehive/certs/chain.pem"

ES Config:

xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.key: /etc/elasticsearch/certs/private.key
xpack.security.transport.ssl.certificate: /etc/elasticsearch/certs/server.crt
xpack.security.transport.ssl.certificate_authorities: [ "/etc/elasticsearch/certs/chain.crt" ]
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.verification_mode: certificate
xpack.security.http.ssl.key: /etc/elasticsearch/certs/private.key
xpack.security.http.ssl.certificate: /etc/elasticsearch/certs/server.crt
xpack.security.http.ssl.certificate_authorities: [ "/etc/elasticsearch/certs/chain.crt" ]

TheHive Log:

[info] o.e.ErrorHandler - GET /api/user/current returned 500
org.elasticsearch.client.transport.NoNodeAvailableException: None of the configured nodes are available: [{#transport#-1}{Ujp2a8HTSPCXc2D2AruT-A}{10.200.204.40}{10.200.204.40:9300}, {#transport#-2}{IboT4FvrSsyvBs87xz5tAQ}{10.200.204.41}{10.200.204.41:9300}, {#transport#-3}{ggxyEsAZQZSAe65YbPTdVw}{10.200.204.42}{10.200.204.42:9300}]
        at org.elasticsearch.client.transport.TransportClientNodesService.ensureNodesAreAvailable(TransportClientNodesService.java:347)
        at org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:245)
        at org.elasticsearch.client.transport.TransportProxyClient.execute(TransportProxyClient.java:59)
        at org.elasticsearch.client.transport.TransportClient.doExecute(TransportClient.java:363)
        at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:408)
        at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:80)
        at com.sksamuel.elastic4s.search.SearchImplicits$SearchDefinitionExecutable$.$anonfun$apply$1(SearchImplicits.scala:27)
        at com.sksamuel.elastic4s.search.SearchImplicits$SearchDefinitionExecutable$.$anonfun$apply$1$adapted(SearchImplicits.scala:27)
        at com.sksamuel.elastic4s.Executable.injectFutureAndMap(Executable.scala:21)
        at com.sksamuel.elastic4s.Executable.injectFutureAndMap$(Executable.scala:19)

ES Log:

_[2018-09-03T00:00:01,214][WARN ][o.e.x.s.t.n.SecurityNetty4Transport] [SOC-sl00z2] exception caught on transport layer [[id: 0x29494870, L:0.0.0.0/0.0.0.0:9300 ! R:/10.200.204.42:33652]], closing connection
io.netty.handler.codec.DecoderException: io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record: 45530000005800000000000fb07808004c4ba3010d417574686f72697a6174696f6e224261736963205a57786863335270597a706c6247467a64476c6a58314e50517a45340016696e7465726e616c3a7463702f68616e647368616b6500
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:459) ~[netty-codec-4.1.13.Final.jar:4.1.13.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265) ~[netty-codec-4.1.13.Final.jar:4.1.13.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
        at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:134) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:644) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:544) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:498) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:458) [netty-transport-4.1.13.Final.jar:4.1.13.Final]
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) [netty-common-4.1.13.Final.jar:4.1.13.Final]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131]
Caused by: io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record: 45530000005800000000000fb07808004c4ba3010d417574686f72697a6174696f6e224261736963205a57786863335270597a706c6247467a64476c6a58314e50517a45340016696e7465726e616c3a7463702f68616e647368616b6500
        at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1103) ~[?:?]
        at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:489) ~[?:?]
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:428) ~[?:?]
        ... 15 more_
kara-1234 commented 5 years ago

As far as I can tell TheHive is trying to communicate without SSL/TLS on.

nadouani commented 5 years ago

Does xpack work without the SSL config? the errors don't seem to be related to xpack

kara-1234 commented 5 years ago

Putting in the search username and password shows no error in elastic, but thehive has the following error.

2019-02-26 14:12:39, 342 [INFO] from org.elasticsearch.client.transport.TransportclientNodesService in elasticsearch[_client_][generic][T#3] - failed to get not info for [#transport#-1][Z6svhk8QiimhjMpX8NQ] {xx.xx.xx.xx:9300}, disconnecting...
org.elasticsearch.transport.ReportTransportException: [hive-1][xxx.xxx.xxx.xxx:9300][cluster:monitor/nodes/liveness]
Caused by: org.elasticsearch.ElasticserchSecurityException: missing authentication token for action [cluster:monitor/nodes/liveness]
    at org.elasticsearch.xpack.security.suporrt.Exceptions.authenticationError(Exceptions.java:39)
    at org.elasticsearch.xpack.security.authc.DefaultAuthenticationFailureHandler.missingToken(DefaultAuthenticationFailureHandler.java:74)
    at org.elasticsearch.xpack.security.authc..AuthenticationService$AuditableTransportRequest.anonymousAccessDenied(AuthenticationService.java553)

I'm at a loss. =/

infde6 commented 5 years ago

I am also experiencing this problem with X-Pack authentication. Was a solution posted somewhere? (Google didn't return anything more relevant than this.

Steps to reproduce:

application.conf entries: index = [index name] cluster = hive host = ["127.0.0.1:9300"] search.username = "username" search.password = "password" search.ssl.enabled = false

thehive errors: [info] o.e.c.t.TransportClientNodesService - failed to get node info for {#transport#-1}{hPf2z2MoSDq8kK5zH9MSgQ}{127.0.0.1}{127.0.0.1:9300}, disconnecting... org.elasticsearch.transport.RemoteTransportException: [node1][127.0.0.1:9300][cluster:monitor/nodes/liveness] Caused by: org.elasticsearch.ElasticsearchSecurityException: missing authentication token for action [cluster:monitor/nodes/liveness] at org.elasticsearch.xpack.core.security.support.Exceptions.authenticationError(Exceptions.java:18) at org.elasticsearch.xpack.core.security.authc.DefaultAuthenticationFailureHandler.createAuthenticationError(DefaultAuthenticationFailureHandler.java:163) at org.elasticsearch.xpack.core.security.authc.DefaultAuthenticationFailureHandler.missingToken(DefaultAuthenticationFailureHandler.java:118) at org.elasticsearch.xpack.security.authc.AuthenticationService$AuditableTransportRequest.anonymousAccessDenied(AuthenticationService.java:658) at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$handleNullToken$19(AuthenticationService.java:467) at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.handleNullToken(AuthenticationService.java:472) at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.consumeToken(AuthenticationService.java:356) at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$extractToken$9(AuthenticationService.java:327) at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.extractToken(AuthenticationService.java:345) at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$checkForApiKey$3(AuthenticationService.java:288)

Elasticsearch does not produce any errors.

Thank you in advance if you can provide any insight / assistance to get this working.

1earch commented 5 years ago

Same issue as #1046 I think :wink: