search

dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
15.07k stars 4.69k forks source link

System.Net.Security.Tests.SslStreamTlsResumeTests failing #103354

Closed kunalspathak closed 2 months ago

kunalspathak commented 3 months ago

Build Information

Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=705166 Build error leg or test failing: System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds Pull request: https://github.com/dotnet/runtime/pull/103159

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": ["ClientCert_DefaultValue_ResumeSucceeds", "Assert.Equal() Failure: Values differ"],
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Mostly due to https://github.com/dotnet/runtime/pull/102656 cc: @rzikm

Known issue validation

Build: :mag_right: https://dev.azure.com/dnceng-public/public/_build/results?buildId=705166 Error message validated: [ClientCert_DefaultValue_ResumeSucceeds Assert.Equal() Failure: Values differ] Result validation: :white_check_mark: Known issue matched with the provided build. Validation performed at: 6/13/2024 7:49:03 AM UTC

Report

Build Definition Test Pull Request
705793 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#101095
705631 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103369
706214 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103382
706355 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103226
705744 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#102834
706320 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#102523
706349 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103399
706344 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103375
706113 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#102736
706298 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#101969
706149 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103143
706263 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103331
705570 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103332
705479 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103241
706227 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103394
706187 dotnet/runtime System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_Proxy_Test.MultiProxy_PAC_Failover_Succeeds dotnet/runtime#103364
706174 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientDisableTlsResume_Succeeds dotnet/runtime#103392
706155 dotnet/runtime JIT/opt/ObjectStackAllocation/ObjectStackAllocationTests/ObjectStackAllocationTests.cmd dotnet/runtime#103361
706094 dotnet/runtime JIT/Regression/JitBlue/GitHub_18770/GitHub_18770/GitHub_18770.dll dotnet/runtime#103382
706138 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103391
706088 dotnet/runtime JIT/Regression/JitBlue/GitHub_18770/GitHub_18770/GitHub_18770.dll dotnet/runtime#103382
706127 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103390
706061 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103387
705940 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103206
706039 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103358
706025 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103386
705911 dotnet/runtime WasmTestOnV8-ST-System.Runtime.Tests.WorkItemExecution dotnet/runtime#103381
705977 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103076
705958 dotnet/runtime System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_Proxy_Test.MultiProxy_PAC_Failover_Succeeds dotnet/runtime#103364
705952 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#102364
705662 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#99183
705881 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103375
705762 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#100334
705726 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103374
705672 dotnet/runtime JIT/opt/ObjectStackAllocation/ObjectStackAllocationTests/ObjectStackAllocationTests.cmd dotnet/runtime#103361
705719 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103322
705463 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103356
705559 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103279
705536 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103358
705451 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#101876
705416 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103346
705403 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103058
705367 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103352
705344 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#102882
705232 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103195
705321 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103191
705312 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#100356
705283 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103346

Summary

24-Hour Hit Count 7-Day Hit Count 1-Month Count
0 0 48
dotnet-policy-service[bot] commented 3 months ago

Tagging subscribers to this area: @dotnet/ncl, @bartonjs, @vcsjones See info in area-owners.md if you want to be subscribed.

rzikm commented 3 months ago

hmm, looks like TLS Resume is not working on TLS 1.3 on Ubuntu 18.04? I will take a look

rzikm commented 3 months ago

@v-wenyuxu is it really the same issue? The test have been disabled, and I don't see this failure in the text of your comment.

rzikm commented 2 months ago

Added some debug print on the related platform and there seems to be something fishy happening in OpenSSL

helixbot@77d5709caac1:/test$ ./dotnet exec --runtimeconfig System.Net.Security.Tests.runtimeconfig.json --depsfile System.Net.Security.Tests.deps.json xunit.console.dll System.Net.Security.Tests.dll -xml testResults.xml -nologo -notrait category=OuterLoop -notrait category=failing -method *NoClientCert_DefaultValue_ResumeSucceeds
  Discovering: System.Net.Security.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  System.Net.Security.Tests (found 1 of 514 test case)
  Starting:    System.Net.Security.Tests (parallel test collections = on [8 threads], stop on fail = off)
Session reused: False
Session reused: False
Adding session to dade675368e24722aa72162292629e68
Adding session to dade675368e24722aa72162292629e68
removing session to dade675368e24722aa72162292629e68
Session reused: False
Session reused: False
setting session
removing session to dade675368e24722aa72162292629e68
Session reused: True
Session reused: True
Adding session to dade675368e24722aa72162292629e68
Session reused: True
Session reused: True
setting session
removing session to @_f�
Session reused: True
Session reused: True
Adding session to @_f�
Session reused: True
Session reused: True
setting session
Session reused: False
Session reused: False
    System.Net.Security.Tests.SslStreamTlsResumeTests.NoClientCert_DefaultValue_ResumeSucceeds(sslProtocol: Tls13) [FAIL]
      Assert.Equal() Failure: Values differ
      Expected: True
      Actual:   False
rzikm commented 2 months ago

The same tests work fine on Ubuntu 20.04 ARM, I believe this is due to a bug in OpenSSL, the version in 18.04 image is stuck on OpenSSL 1.1.1 (no letter in the version) and the image does not receive updates due to being out of support. I think we should keep the problematic combination disabled for this test.