TheHive-Project / TheHive

TheHive: a Scalable, Open Source and Free Security Incident Response Platform
https://thehive-project.org
GNU Affero General Public License v3.0
3.39k stars 618 forks source link

[Bug] Regression starting with 4.1.17 in the migration tool with certificate validation #2342

Closed andres-tw closed 2 years ago

andres-tw commented 2 years ago

Request Type

Bug

Work Environment

Question Answer
OS version (server) Ubuntu
OS version (client)
Virtualized Env. True
Dedicated RAM 8 GB
vCPU 4
TheHive version / git hash >= 4.1.17-1
Package Type Docker
Database Cassandra
Index type Elasticsearch
Attachments storage S3
Browser type & version If applicable

Problem Description

Starting with version 4.1.17 when running the migration tool in a docker container initialisation of the migration fails with PKIX path building failed. Exactly the same configs work for starting the migration in 4.1.16-1.

Steps to Reproduce

  1. Working configs for Hive 4 & Hive 3.5.1
  2. docker pull thehiveproject/thehive4:4.1.17-1
  3. docker run -it --entrypoint /bin/bash -v "/opt/app/thehive/hive3.conf:/etc/thehive/hive3.conf:rw" -v "/opt/app/thehive/application.conf:/etc/thehive/application.conf:rw" -v "/opt/app/thehive/es.jks:/etc/thehive/es.jks:rw" --network host docker.tw.ee/tw-secops-thehive
  4. Run migration tool: ./bin/migrate -d --output /etc/thehive/application.conf --main-organisation ORG_NAME --input /etc/thehive/hive3.conf

Possible Solutions

Not a solution, but a workaround might be to enable index.search.elasticsearch.ssl.allow-self-signed-certificates for migration connections somehow.

Complementary information

Full stack trace:

[error] Migration failed
java.net.ConnectException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at play.shaded.ahc.org.asynchttpclient.netty.channel.NettyConnectListener.onFailure(NettyConnectListener.java:179)
    at play.shaded.ahc.org.asynchttpclient.netty.channel.NettyConnectListener$1.onFailure(NettyConnectListener.java:151)
    at play.shaded.ahc.org.asynchttpclient.netty.SimpleFutureListener.operationComplete(SimpleFutureListener.java:26)
    at play.shaded.ahc.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:577)
    at play.shaded.ahc.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:570)
    at play.shaded.ahc.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:549)
    at play.shaded.ahc.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:490)
    at play.shaded.ahc.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:615)
    at play.shaded.ahc.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:608)
    at play.shaded.ahc.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:117)
Caused by: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at sun.security.ssl.Alert.createSSLException(Alert.java:131)
    at sun.security.ssl.TransportContext.fatal(TransportContext.java:324)
    at sun.security.ssl.TransportContext.fatal(TransportContext.java:267)
    at sun.security.ssl.TransportContext.fatal(TransportContext.java:262)
    at sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:654)
    at sun.security.ssl.CertificateMessage$T12CertificateConsumer.onCertificate(CertificateMessage.java:473)
    at sun.security.ssl.CertificateMessage$T12CertificateConsumer.consume(CertificateMessage.java:369)
    at sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:377)
    at sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:444)
    at sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:981)
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:456)
    at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:323)
    at sun.security.validator.Validator.validate(Validator.java:271)
    at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:315)
    at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:278)
    at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:141)
    at sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:632)
    at sun.security.ssl.CertificateMessage$T12CertificateConsumer.onCertificate(CertificateMessage.java:473)
    at sun.security.ssl.CertificateMessage$T12CertificateConsumer.consume(CertificateMessage.java:369)
    at sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:377)
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141)
    at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126)
    at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280)
    at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:451)
    at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:323)
    at sun.security.validator.Validator.validate(Validator.java:271)
    at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:315)
    at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:278)
    at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:141)
    at sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:632)
[info] Stage: initialisation
cyberpescadito commented 2 years ago

thanks for this report @andres-tw , this will be fixed in 4.1.19

To-om commented 2 years ago

@andres-tw create a configuration file input.conf containing:

search.trustStore.path=/etc/thehive/es.jks
search.trustStore.type=JKS

Then add the parameter --input /path/to/input.conf to the migration tool.

andres-tw commented 2 years ago

@To-om Ah yes, sorry forgot to include the hive3 config. For all tests it's been configured as follows:

play.http.secret.key = "xxxx"

search {
  index = "the_hive"
  # NB! There is currently no way to disable hostname verification for TLS connections to Elastic
  uri = "https://instance:9200"
  nbreplicas = 1
  nbshards = 10
  user = "thehive"
  password = "xxxxx"
  # For SSL to work with ES, both trustStore and keyStore need to be defined
  search.trustStore {
    path = "/etc/thehive/es.jks"
    type = "JKS"
    password = "xxxx"
  }
  search.keyStore {
    path = "/etc/thehive/es.jks"
    type = "JKS"
    password = "xxxx"
  }
}

cluster {
  name = "xxxxx"
}

auth {
  provider = [local]
}

session {
  warning = 5m
  inactivity = 1h
}

# Max textual content length
play.http.parser.maxMemoryBuffer= 10M
# Max file size
play.http.parser.maxDiskBuffer = 1G

Tested now again, with only the trustStore configuration like in your example.

andres-tw commented 2 years ago

@To-om Anything else I can test or your working on it?

To-om commented 2 years ago

You can now configure http client by adding wsConfig section in search. You can find available settings in the documentation here and here. If you want to disable SSL certificate verification you can set:

search {
  index = "the_hive"
  uri = "https://instance:9200"
  wsConfig.ssl.loose.acceptAnyCertificate = true
andres-tw commented 2 years ago

So the new config option did not help. Fails with the same error message on both 4.19.1-1 and 5.0.2 docker images. Tried adding that config option both together with a truststore and without. I also tried configuring the trust store debugging, but that did not produce any extra logs to help me in debugging this further. Could it somehow be related to the fact that I'm trying to do the migration from a docker container?

To-om commented 2 years ago

There is a typo in the last commit. The wsConfig is not search in the correct section. You should adapt the configuration file and put it in search.trustStore like that:

search {
  index = "the_hive"
  uri = "https://instance:9200"
  trustStore.wsConfig.ssl.loose.acceptAnyCertificate = true

This will be fixed in the next release.