search

fido-alliance / conformance-test-tools-resources

Certification Test Tools Resources. For security and privacy related issues email tools@certification.fidoalliance.org
https://fidoalliance.org/certification/functional-certification/conformance/
43 stars 14 forks source link

Test results are unpredictable :) #392

Closed dawid-nowak closed 6 years ago

dawid-nowak commented 6 years ago

By submitting this issue you are acknowledging that any information regarding this issue will be publicly available.

If you have privacy concerns, please email conformance-tools@fidoalliance.org

What protocol and version of the protocol are you testing?

UAF/U2F/FIDO2 v1.0/v1.1/v1.2 FIDO2

What is your implementation class?

Server/Client/ASM+Authr

What is the version of the tool are you using?

FIDO Conformance Tools v0.10.109 (BETA)((BETA) FIDO2 )

What is the OS and the version are you running?

Windows 7

Issue description

Running MDS test Server-ServerAuthenticatorAttestationResponse-Resp-1 Test server processing ServerAuthenticatorAttestationResponse structure. F-5 Send a valid ServerAuthenticatorAttestationResponse with FULL "packed" attestation for metadata from MDS who's metadata service intermediate certificate is revoked, and check that

Conformance tool fetches the data from MDS test server. The data seems to be different with every attempt. In our case this leads to unpredictable results and the server sometimes passes the test and sometimes it doesn't.

I would expect that passing certification should not be dependent on luck.

The screen shot shows to attempts against the server and one is passing and one is not.

image

dawid-nowak commented 6 years ago

I am thinking this could be due to our beloved security policies