search

elastic / elasticsearch

Free and Open Source, Distributed, RESTful Search Engine
https://www.elastic.co/products/elasticsearch
Other
1.09k stars 24.83k forks source link

[CI] SSLErrorMessageCertificateVerificationTests fails for fips #65373

Closed hendrikmuhs closed 5 days ago

hendrikmuhs commented 3 years ago

The important bit seems to be -Dtests.fips.enabled=true, switch to false make it pass.

Build scan: https://gradle-enterprise.elastic.co/s/xzikwnf4w4m6k

Repro line:

REPRODUCE WITH: ./gradlew ':x-pack:plugin:security:test' --tests "org.elasticsearch.xpack.ssl.SSLErrorMessageCertificateVerificationTests.testMessageForHttpClientHostnameVerificationFailure" \
  -Dtests.seed=AD420F801CD295C7 \
  -Dtests.security.manager=true \
  -Dtests.locale=el-GR \
  -Dtests.timezone=Etc/GMT-6 \
  -Druntime.java=8 \
  -Dtests.fips.enabled=true

REPRODUCE WITH: ./gradlew ':x-pack:plugin:security:test' --tests "org.elasticsearch.xpack.ssl.SSLErrorMessageCertificateVerificationTests.testMessageForRestClientHostnameVerificationFailure" \
  -Dtests.seed=AD420F801CD295C7 \
  -Dtests.security.manager=true \
  -Dtests.locale=el-GR \
  -Dtests.timezone=Etc/GMT-6 \
  -Druntime.java=8 \
  -Dtests.fips.enabled=true

REPRODUCE WITH: ./gradlew ':x-pack:plugin:security:test' --tests "org.elasticsearch.xpack.ssl.SSLErrorMessageCertificateVerificationTests.testMessageForHttpClientHostnameVerificationFailure" \
  -Dtests.seed=AD420F801CD295C7 \
  -Dtests.security.manager=true \
  -Dtests.locale=el-GR \
  -Dtests.timezone=Etc/GMT-6 \
  -Druntime.java=8 \
  -Dtests.fips.enabled=true

REPRODUCE WITH: ./gradlew ':x-pack:plugin:security:test' --tests "org.elasticsearch.xpack.ssl.SSLErrorMessageCertificateVerificationTests.testMessageForRestClientHostnameVerificationFailure" \
  -Dtests.seed=AD420F801CD295C7 \
  -Dtests.security.manager=true \
  -Dtests.locale=el-GR \
  -Dtests.timezone=Etc/GMT-6 \
  -Druntime.java=8 \
  -Dtests.fips.enabled=true

REPRODUCE WITH: ./gradlew ':x-pack:plugin:security:test' --tests "org.elasticsearch.xpack.security.transport.netty4.SimpleSecurityNetty4ServerTransportTests.testTcpHandshake" \
  -Dtests.seed=AD420F801CD295C7 \
  -Dtests.security.manager=true \
  -Dtests.locale=sr-ME \
  -Dtests.timezone=Africa/Bamako \
  -Druntime.java=8 \
  -Dtests.fips.enabled=true

REPRODUCE WITH: ./gradlew ':x-pack:plugin:security:test' --tests "org.elasticsearch.xpack.security.transport.netty4.SimpleSecurityNetty4ServerTransportTests.testAcceptedChannelCount" \
  -Dtests.seed=AD420F801CD295C7 \
  -Dtests.security.manager=true \
  -Dtests.locale=sr-ME \
  -Dtests.timezone=Africa/Bamako \
  -Druntime.java=8 \
  -Dtests.fips.enabled=true

REPRODUCE WITH: ./gradlew ':x-pack:plugin:security:test' --tests "org.elasticsearch.xpack.security.transport.netty4.SimpleSecurityNetty4ServerTransportTests.testConcurrentSendRespondAndDisconnect" \
  -Dtests.seed=AD420F801CD295C7 \
  -Dtests.security.manager=true \
  -Dtests.locale=sr-ME \
  -Dtests.timezone=Africa/Bamako \
  -Druntime.java=8 \
  -Dtests.fips.enabled=true

REPRODUCE WITH: ./gradlew ':x-pack:plugin:security:test' --tests "org.elasticsearch.xpack.security.transport.netty4.SimpleSecurityNetty4ServerTransportTests.testFailToSend" \
  -Dtests.seed=AD420F801CD295C7 \
  -Dtests.security.manager=true \
  -Dtests.locale=sr-ME \
  -Dtests.timezone=Africa/Bamako \
  -Druntime.java=8 \
  -Dtests.fips.enabled=true

REPRODUCE WITH: ./gradlew ':x-pack:plugin:security:test' --tests "org.elasticsearch.xpack.security.transport.netty4.SimpleSecurityNetty4ServerTransportTests.testVersionFrom1to1" \
  -Dtests.seed=AD420F801CD295C7 \
  -Dtests.security.manager=true \
  -Dtests.locale=sr-ME \
  -Dtests.timezone=Africa/Bamako \
  -Druntime.java=8 \
  -Dtests.fips.enabled=true

REPRODUCE WITH: ./gradlew ':x-pack:plugin:security:test' --tests "org.elasticsearch.xpack.security.transport.netty4.SimpleSecurityNetty4ServerTransportTests.testVoidMessageCompressed" \
  -Dtests.seed=AD420F801CD295C7 \
  -Dtests.security.manager=true \
  -Dtests.locale=sr-ME \
  -Dtests.timezone=Africa/Bamako \
  -Druntime.java=8 \
  -Dtests.fips.enabled=true

Reproduces locally?: yes

Applicable branches: 7.x

Failure history:

Failure excerpt:

org.elasticsearch.xpack.ssl.SSLErrorMessageCertificateVerificationTests > testMessageForHttpClientHostnameVerificationFailure FAILED
16:20:51     java.lang.AssertionError: 
16:20:51     Expected: a throwable with message of a string containing "Certificate" ignoring case
16:20:51          but: was <javax.net.ssl.SSLException: Connection reset> at sun.security.ssl.Alert.createSSLException(Alert.java:127)
16:20:51         at __randomizedtesting.SeedInfo.seed([AD420F801CD295C7:AF1435585E4172F4]:0)
16:20:51         at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:18)
16:20:51         at org.junit.Assert.assertThat(Assert.java:956)
16:20:51         at org.junit.Assert.assertThat(Assert.java:923)
16:20:51         at
elasticmachine commented 3 years ago

Pinging @elastic/es-security (Team:Security)

jkakavas commented 3 years ago

Apparently the latest JDK8 updates made it so this fails with a different error and this has been masked because 7.x has been failing in general in FIPS mode ( tracked in https://github.com/elastic/elasticsearch/issues/64379 which will hopefully be fixed by https://github.com/elastic/elasticsearch/issues/64776 )

ywangd commented 3 years ago

Another failure of SimpleSecurityNioTransportTests.testConcurrentSendRespondAndDisconnect for 7.13 with JDK16: https://gradle-enterprise.elastic.co/s/5kevwhgfekf3y

It could be transient since it failed only once in last 7 days. It may also not be related to the original issue raised here. But I am attaching it here since it may be transient and not worth its own issue and it is kinda related to this one.

The error message is

org.elasticsearch.transport.NodeDisconnectedException: [TS_B_6][127.0.0.1:18602][internal:transport/handshake] disconnected

And it logs warnings of:

javax.net.ssl.SSLException: Closed engine without completely sending the close alert message. |  
-- | --
  | at org.elasticsearch.xpack.security.transport.nio.SSLDriver.close(SSLDriver.java:161) ~[main/:?] |  
  | at org.elasticsearch.core.internal.io.IOUtils.close(IOUtils.java:74) ~[elasticsearch-core-7.13.0-SNAPSHOT.jar:7.13.0-SNAPSHOT] |  
  | at org.elasticsearch.core.internal.io.IOUtils.close(IOUtils.java:116) ~[elasticsearch-core-7.13.0-SNAPSHOT.jar:7.13.0-SNAPSHOT] |  
  | at org.elasticsearch.core.internal.io.IOUtils.close(IOUtils.java:66) ~[elasticsearch-core-7.13.0-SNAPSHOT.jar:7.13.0-SNAPSHOT] |  
  | at org.elasticsearch.xpack.security.transport.nio.SSLChannelContext.closeFromSelector(SSLChannelContext.java:207) ~[main/:?] |  
  | at org.elasticsearch.nio.EventHandler.handleClose(EventHandler.java:229) [elasticsearch-nio-7.13.0-SNAPSHOT.jar:7.13.0-SNAPSHOT] |  
  | at org.elasticsearch.nio.EventHandler.postHandling(EventHandler.java:187) [elasticsearch-nio-7.13.0-SNAPSHOT.jar:7.13.0-SNAPSHOT] |  
  | at org.elasticsearch.nio.NioSelector.processKey(NioSelector.java:239) [elasticsearch-nio-7.13.0-SNAPSHOT.jar:7.13.0-SNAPSHOT] |  
  | at org.elasticsearch.nio.NioSelector.singleLoop(NioSelector.java:163) [elasticsearch-nio-7.13.0-SNAPSHOT.jar:7.13.0-SNAPSHOT] |  
  | at org.elasticsearch.nio.NioSelector.runLoop(NioSelector.java:120) [elasticsearch-nio-7.13.0-SNAPSHOT.jar:7.13.0-SNAPSHOT] |  
  | at java.lang.Thread.run(Thread.java:831) [?:?]
jtibshirani commented 2 years ago

The test SimpleSecurityNioTransportTests.testConcurrentSendRespondAndDisconnect failed again recently with a similar error message and the same warnings: https://gradle-enterprise.elastic.co/s/7yd4nadtxacn2

elasticsearchmachine commented 5 days ago

This issue has been closed because it has been open for too long with no activity.

Any muted tests that were associated with this issue have been unmuted.

If the tests begin failing again, a new issue will be opened, and they may be muted again.