Open danmoseley opened 5 years ago
@bartonjs
Failed again:
AssociatePersistedKey_CAPIviaCNG_RSA(provType: 1, keyNumber: Exchange)
AssociatePersistedKey_CAPIviaCNG_RSA(provType: 1, keyNumber: Signature)
AssociatePersistedKey_CAPIviaCNG_RSA(provType: 12, keyNumber: Exchange)
AssociatePersistedKey_CAPIviaCNG_RSA(provType: 24, keyNumber: Exchange)
AssociatePersistedKey_CAPIviaCNG_RSA(provType: 24, keyNumber: Signature)
Message :
Internal.Cryptography.CryptoThrowHelper+WindowsCryptographicException : Keyset as registered is invalid.
Stack Trace :
at Internal.NativeCrypto.CapiHelper.CreateProvHandle(CspParameters parameters, Boolean randomKeyContainer) in /_/src/System.Security.Cryptography.Csp/src/System/Security/Cryptography/CapiHelper.Windows.cs:line 287
at System.Security.Cryptography.RSACryptoServiceProvider.get_SafeProvHandle() in /_/src/System.Security.Cryptography.Csp/src/System/Security/Cryptography/RSACryptoServiceProvider.Windows.cs:line 86
at System.Security.Cryptography.RSACryptoServiceProvider.get_SafeKeyHandle() in /_/src/System.Security.Cryptography.Csp/src/System/Security/Cryptography/RSACryptoServiceProvider.Windows.cs:line 135
at System.Security.Cryptography.RSACryptoServiceProvider..ctor(Int32 keySize, CspParameters parameters, Boolean useDefaultKeySize) in /_/src/System.Security.Cryptography.Csp/src/System/Security/Cryptography/RSACryptoServiceProvider.Windows.cs:line 71
at System.Security.Cryptography.RSACryptoServiceProvider..ctor(CspParameters parameters) in /_/src/System.Security.Cryptography.Csp/src/System/Security/Cryptography/RSACryptoServiceProvider.Windows.cs:line 47
at System.Security.Cryptography.X509Certificates.Tests.CertificateCreation.PrivateKeyAssociationTests.AssociatePersistedKey_CAPIviaCNG_RSA(Int32 provType, KeyNumber keyNumber) in /_/src/System.Security.Cryptography.X509Certificates/tests/CertificateCreation/PrivateKeyAssociationTests.cs:line 97
Windows ARM64 isn't officially supported in 3.0
I will disable the test meanwhile on arm64.
@ViktorHofer Is Win-ARM64 officially supported yet? If not, the run is informative only, and shouldn't count toward CI council metrics (and tests shouldn't be disabled).
Is Win-ARM64 officially supported yet?
The support matrix for 3.0 can be found here: https://github.com/dotnet/core/blob/master/release-notes/3.0/3.0-supported-os.md. There is no announcement regarding the support matrix for 5.0 yet. That said I assume we will support arm64 in some way in 5.0 therefore this belongs into master.
shouldn't count toward CI council metrics (and tests shouldn't be disabled).
The CI council looks at the build pass-rate which includes tests. We don't differentiate between configurations in a build. This isn't just about the CI council but about producing green builds. If a test is flaky it should either be fixed or disabled, regardless of its configuration.
cc @stephentoub
If we're running the tests routinely on a given platform, then we're obviously doing work to bring up that platform. If we're not doing work to bring-up the platform, then we shouldn't be running any tests at all. If we are doing work to bring-up the platform, then known failures are noise that inhibit further investigations, so why wouldn't we disable tests known to be problematic until they can be investigated?
It's all about the circular position really.
If we're not doing work to bring-up the platform, then we shouldn't be running any tests at all.
I agree. If we're planning on supporting it for 5.0 then things are actionable. I've heard no such statement, so (IMO) we should disable the leg. Or ignore it.
so why wouldn't we disable tests known to be problematic until they can be investigated?
Mainly, that I (at least) don't have the time to spend on investigating things on unsupported platforms. Until it's a platform we're actually going to support it's not worth investigating (particularly since I don't know how to do step 1: obtain a repro environment).
If it's a flaky test then we can't discern if it's "fixed by some unknown change" (us or the OS) or "locally passing, but will still fail right after re-enabling".
But, ultimately, since the bug will still be open and tracking it, I guess... whatever. (Mainly my reaction was "don't just suppress problems, it'll make it harder when we want to actually support the problem and we don't realize our debt")
If it's a flaky test then we can't discern if it's "fixed by some unknown change" (us or the OS) or "locally passing, but will still fail right after re-enabling".
If the test is known to fail, then everyone just learns to ignore it. No one will notice when it stops failing because everyone becomes blind to it. And yet it becomes noise in any output listings... if you've got 100 failures, 99 of which are noise, you're still forced to review every failure to determine whether it's known or not, in order to determine whether any are not, and that takes non-trivial time.
Looks like this may be sporadic.
https://mc.dot.net/#/product/netcore/30/source/official~2Fdotnet~2Fcorefx~2Frefs~2Fheads~2Fmaster/type/test~2Ffunctional~2Fcli~2F/build/20190324.4/workItem/System.Security.Cryptography.X509Certificates.Tests/analysis/xunit/System.Security.Cryptography.X509Certificates.Tests.CertificateCreation.PrivateKeyAssociationTests~2FAssociatePersistedKey_CAPIviaCNG_RSA(provType:%201,%20keyNumber:%20Exchange)