search

openid-certification / oidctest

THE CERTIFICATION TEST SUITE HAS BEEN MIGRATED TO A NEW SERVICE https://www.certificatinon.openid.net
Other
50 stars 15 forks source link

conformance profiles document out of sync with test suite #49

Open zandbelt opened 7 years ago

zandbelt commented 7 years ago

[1] is the documentation I refer to. I have run only the basic profile. The mismatches I found in basic profile are given below.

Included in the test suit but not in the documentation

OP-Response-form_post
Included in the documentation but not in the test suit

OP-request_uri-Unsigned OP-ClientAuth-Basic-Dynamic OP-ClientAuth-SecretPost-Dynamic OP-IDToken-none OP-IDToken-kid OP-IDToken-RS256

[1] - http://openid.net/wordpress-content/uploads/2016/12/OpenID-Connect-Conformance-Profiles.pdf

I believe all tests are still there, they just have different names. Changing the names of the tests may result in problems for testers using automated certification, so we are going to change the names in the doc.

I'll report back here which tests have been renamed to which names asap.

selfissued commented 7 years ago

Hans will send the changes to Mike, who will then update the documents.

zandbelt commented 7 years ago

The following:

I believe the tester is missing those tests in the list because of the configuration of the test tool instance; the following should be enabled:

Cryptographic support: signing none encryption

zandbelt commented 7 years ago

The tester confirmed: "Yeah I think it is due to the configuration." Now there's just the matter of OP-Response-form_post not being present in the document yet but it is optional anyhow (and we'll have more like op-init-sso) so I believe we can close this for now.

selfissued commented 6 years ago

Both and OP-request_uri-Support and OP-Rotation-OP-Sig are currently displayed even when not testing the Dynamic OP profile, but they should only appear for Dynamic - per the profile definitions at http://openid.net/wordpress-content/uploads/2016/12/OpenID-Connect-Conformance-Profiles.pdf . @rohe - can you do an updated release restricting their visibility to Dynamic (and possibly Extra). Thanks.

Note that this is not a complete review of the potential issues on my part. It's just cases I know of because they are causing confusion for testers.

zandbelt commented 5 years ago

Also rp-claims-distributed and rp-claims-aggregated are in the list of RP tests but not reflected in the latest version of the conformance profiles doc: https://openid.net/wordpress-content/uploads/2018/06/OpenID-Connect-Conformance-Profiles.pdf