dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
15.27k stars 4.73k forks source link

Two X509 Pfx test failures on OS X #15067

Closed stephentoub closed 4 years ago

stephentoub commented 9 years ago

I get these when running locally on OS X:

Discovering: System.Security.Cryptography.X509Certificates.Tests
Discovered:  System.Security.Cryptography.X509Certificates.Tests
Starting:    System.Security.Cryptography.X509Certificates.Tests
   System.Security.Cryptography.X509Certificates.Tests.CollectionTests.ExportUnrelatedPfx [FAIL]
      Assert.Equal() Failure
      Expected: String[] ["Microsoft Corporation", "cn.subject.example.org", "Foo"]
      Actual:   String[] ["Foo", "cn.subject.example.org", "Microsoft Corporation"]
      Stack Trace:
            at System.Security.Cryptography.X509Certificates.Tests.CollectionTests.ExportUnrelatedPfx()
   System.Security.Cryptography.X509Certificates.Tests.CollectionTests.ImportFullChainPfx [FAIL]
      Assert.Equal() Failure
      Expected: String[] ["MS Passport Test Sub CA", "MS Passport Test Root CA", "test.local"]
      Actual:   String[] ["MS Passport Test Root CA", "MS Passport Test Sub CA", "test.local"]
      Stack Trace:
            at System.Security.Cryptography.X509Certificates.Tests.CollectionTests.ImportFullChainPfx()
Finished:    System.Security.Cryptography.X509Certificates.Tests

Looks like some data is ending up in an unexpected order.

stephentoub commented 9 years ago

@bartonjs, can you take a look at this one? This is failing consistently and is (one of the tests) preventing a clean run on OS X. I looked at the tests, and at least one of them implies the ordering should be consistent across platforms, so I'm not sure if it's just a test bug (i.e. we should ignore ordering in the test) or if it's a product bug.

bartonjs commented 9 years ago

@stephentoub Odd, I disabled one of the tests, but not the other. I've updated them both to be order-independent verification; and opened dotnet/corefx#3207 to track that the ordering isn't copacetic.