search

openid / OpenID4VP

58 stars 20 forks source link

Certification/Conformance testing for OID4VC (starting with testing wallets in 4VP) #3

Open OIDF-automation opened 2 years ago

OIDF-automation commented 2 years ago

Imported from AB/Connect bitbucket: https://bitbucket.org/openid/connect/issues/1464

Original Reporter: josephheenan

Gail has asked if I could estimate the work involved in creating conformance/certification tests for SIOP & verifiable presentations, in similar ways to the tests created for OpenID Connect / FAPI / FAPI-CIBA / etc.

Before I do that, it would be great to get some input from the working group.

  1. Are there test systems we can test certification tests against? Ideally these would be sandbox-type systems that contain no real user data, and where any required user interactions can be automated (this is so we can run automated testing of the tests. We have an existing system for automating simple web interactions along the ‘enter text into this field’ and ‘press this button’.) An example client that shows exactly what is any requests/responses/redirects would also be helpful.
  2. Are there any particular happy-flow or negative scenarios the WG feel are particularly important to test? (For example, for OpenID Connect certification there is a happy flow that requires response_type=code&scope=openid&… to work and return a fully valid id_token, and a negative flow that requires unregistered redirect uris are rejected.)
  3. Any guidance on expected certification profiles and optional/mandatory features would be helpful. (For example, OpenID Connect has a ‘Dynamic’ certification profile that requires the OP to publish authorization server metadata and to support dynamic client registration, and a ‘Basic’ certification profile that requires neither.)
  4. Once we have the above, it’d be good to get confirmation there’s at least 3 OP implementations that at least come close to meeting the requirements and are willing to test the tests once a beta version is available

I’ve assumed for now that tests for identity providers are more important than tests for relying parties, as that’s generally how other working groups have viewed it, but please say if this assumption is wrong.

For example, this is some rough guidance that the ekyc-ida working group provided (with some input from the certification team): https://docs.google.com/document/d/1SX2_SjcMUQJ6SQEuNrhNjqAqpQjTUqkHl_qCReSv9-Y/edit#heading=h.cuz9mnx958lj

OIDF-automation commented 2 years ago

Imported from AB/Connect bitbucket - Original Commenter: tlodderstedt

re 1) I will reach out to one of our partners that has a web wallet with OIDC4SSI support whether that could be used for testing the test

OIDF-automation commented 2 years ago

Imported from AB/Connect bitbucket - Original Commenter: mbj

As I mentioned on last week’s call, in the OpenID Certification program, it's up to the working group to define what to test. Joseph was right to be asking us to do so.

I believe that even a minimal amount of conformance testing could substantially improve the prospects for interoperability. We can start small and add to the set of tests as they make sense.

One of the first decisions to make is whether to start with testing OPs or RPs.

OIDF-automation commented 2 years ago

Imported from AB/Connect bitbucket - Original Commenter: tlodderstedt

I think we should start with OPs (SIOPs) to set a baseline.

OIDF-automation commented 2 years ago

Imported from AB/Connect bitbucket - Original Commenter: KristinaYasuda

few relevant reference links:

‌

OIDF-automation commented 2 years ago

Imported from AB/Connect bitbucket - Original Commenter: mbj

Will you be at IIW, Joseph? If so, let’s dedicate some time together to this topic!

OIDF-automation commented 2 years ago

Imported from AB/Connect bitbucket - Original Commenter: josephheenan

Unfortunately not - I’ll be at OSW, EIC and identiverse.

OIDF-automation commented 2 years ago

Imported from AB/Connect bitbucket - Original Commenter: josephheenan

Here’s the initial document Kristina wrote up on SIOP certification at OSW (May 2022), with some help/hinderance from Torsten & I:

https://docs.google.com/document/d/10ApJGcnygJRqWR-iGpplmqMWdjjylBQanx07Iz18eZo/edit?usp=sharing

OIDF-automation commented 1 year ago

Imported from AB/Connect bitbucket - Original Commenter: josephheenan

Latest doc produced when Kristina / Torsten / I met at IETF Yokohama:

‌

https://docs.google.com/document/d/1CVuFaPX_W7tsjfhhHGuOzoD4ettd1IWWhw1przqv-TU/edit

Sakurann commented 4 months ago

@jogu, I'll assign this to you. Please keep us informed about the status, progress of the conformance testing/certification

Sakurann commented 1 week ago

@jogu maybe you could add pointers to the wallet tests that are already available and verifier tests that will be available soon? or we should close this issue if you don't have time to keep it up to date.

jogu commented 4 days ago

The details of the currently available tests for wallets & verifiers for OID4VP are available here:

https://openid.net/certification/conformance-testing-for-openid-for-verifiable-presentations/

We/I am trying to keep that up to date, mostly successfully.

I think we probably could close this issue, I'm not sure I see a use in keeping it open.

We should schedule an update on the conformance tests into a working group meeting agenda in the relatively near feature I think.

Sakurann commented 2 days ago

ok, let's take time in the WG call to walk through the conformance test details! and close this issue in a week if no objections