search

ConsumerDataStandardsAustralia / standards

Work space for data standards development in Australia under the Consumer Data Right regime
Other
321 stars 56 forks source link

Decision Proposal 108 - Banking Maintenance Iteration 3 #108

Closed CDR-API-Stream closed 4 years ago

CDR-API-Stream commented 4 years ago

This decision captures the outcome of Banking Maintenance Iteration 3. Scope under consideration and consultation for this maintenance iteration is detailed here.

The final decision document has been reviewed and approved by the Data Standards Chair. It is attached below: Decision 108 - Banking Maintenance Iteration 3.pdf.

This change will be included in the next Consumer Data Standards release v1.4.0.

Meeting series

Note: To receive calendar invites for this series please email your request via our mailbox. To receive notifications of this meeting series and more subscribe to the DSB mailing list for regular updates.

Previous Meeting Minutes

Schedule

Phase 1: Backlog Grooming - 6th April 2020 commencement. 2 weeks duration Phase 2: Consultation - 20th April 2020 commencement. 4 weeks duration Phase 3: Approval - 18th May 2020 commencement. 1 week duration Phase 4: Documentation - 25th May 2020 commencement. 1 week duration

Meeting 1, Maintenance Iteration 3: Kickoff and backlog review Time: 2pm AEST, 9th April Duration: 1 hour

Meeting 2, Maintenance Iteration 3: Discuss proposals

Time: 2pm AEST, 14th May Duration: 1 hour

Meeting 3, Maintenance Iteration 3: Retrospective & Maintenance Iteration 4: Kickoff and backlog review

Time: 2pm AEST, 4th June Duration: 1 hour

CDR-API-Stream commented 4 years ago

Meeting notes have been added from the Kickoff & Backlog Grooming meeting.

Feedback on change request prioritisation remains open until COB 19/04/2020.

Michael-Backhouse commented 4 years ago

I've seen a few comments around the Swagger/Open API Spec as not be a source of truth for the data standards and the site should be consulted.
Most modern development is contract first, allowing error-free generation of source, test cases and even initial validation of input. But this approach relies on the OpenAPI spec being an accurate representation of the standard.
Can we get a clarification if this is the case - that the swagger is a "source-of-truth", and, if it isn't can this discussion take place as part of this maintenance release. The proposal is each maintenance release, produces a "source-of-truth" OpenAPI artefact, as well as the release documentation.

deboraelkin2 commented 4 years ago

I've seen a few comments around the Swagger/Open API Spec as not be a source of truth for the data standards and the site should be consulted. Most modern development is contract first, allowing error-free generation of source, test cases and even initial validation of input. But this approach relies on the OpenAPI spec being an accurate representation of the standard. Can we get a clarification if this is the case - that the swagger is a "source-of-truth", and, if it isn't can this discussion take place as part of this maintenance release. The proposal is each maintenance release, produces a "source-of-truth" OpenAPI artefact, as well as the release documentation.

I agree that having up-to-date OpenAPI specs should be prioritised. There are plenty of tools that automatically generate interactive documentation and test sandboxes from Open API specifications. In order to make use of such tools, we now have to manually correct the OpenAPI specification (eg: to include security schemas, add missing parameters, etc.)

It would also help, I think, to have, not a monolithic Open API specification, but broken down in smaller areas (eg: products, accounts, balances, etc), this would improve the rendering for the interactive documentation.

I personally created some tools to break down the specification into smaller files, but it'd be much simpler to have them from the source, so to speak.

CDR-API-Stream commented 4 years ago

Thanks @Michael-Backhouse the definition of source of truth varies depending on intention. For the Data Standards, the underlying source of truth are the decision records themselves. The standards documentation and Swagger specification are developed to those decisions. Because the CDR is not purely an IT project, but a government reform project, the Swagger cannot be the only consideration.

Equally the reason that the Swagger specification is not treated as the source of truth is because not all constraints and requirements can be expressed purely in the Swagger/OpenAPI language. This is why the standards documentation should be considered in totality as it allows for further detail where the Swagger language does not allow.

CDR-API-Stream commented 4 years ago

Hi @deboraelkin2 you raise some good points. The DSB is currently assessing OAS3 and ReDoc as well as refactoring the Swagger specification to be more modular. The view is to better provide a more manageable specification over time and address some of the points you raise. Also note that OAS3 has improvements to the API specification language features that are not available with Swagger 2.

Previously the DSB had consulted on the use of OAS 3 but at the time the feedback we received was that many API Gateways did not then have support for OAS 3. If that has changed and their is broader support it would make the move to OAS 3 more feasible.

perlboy commented 4 years ago

@deboraelkin2 For what it's worth the CDS Codegen has a Swagger generator in it that is based on the API Model. It's also template driven (Mustache now) so changes can be incorporated there. In addition, the CDS API Model was designed to support all of the constraints that @CDR-API-Stream has mentioned.

Suffice to say, when it existed, the Engineering team proposed the adoption of Codegen generated Swaggers allowing for a single source of truth driven off the API Model library. This was proposed multiple times within the DSB but was rejected primarily because the spec maintainers didn't want to write Java POJO's or produce backlogs for Engineering to perform the translation. Now... here we are...

@CDR-API-Stream Deep Thought already generates the entire CDS specification in OAS3 and Biza already publishes a Postman collection which incorporates all the headers and auth. FWIW OpenAPI 3 has support for Hybrid Flow but Swagger UI doesn't and even if it did the CDR modifications mean they will never work.

Nonetheless, you're welcome.

perlboy commented 4 years ago

Oh, final side note, splitting up of OpenAPI specs "seems" like a good idea but can get very very messy very quickly and many tools will barf when they encounter it. Additionally, the tools that do support it have quite a few issues resulting in it being very easy to produce broken specs without realising it.

Combine this with versioning requirements, it became clear that, outside of defining a DSL which Engineering would have preferred had it not been for time constraints, a POJO approach provided the ability to do compile checked splitting of definition with the OpenAPI spec being the product. We also proposed that the Slate documentation eventually be generated from this same source of truth but this was also rejected....

deboraelkin2 commented 4 years ago

Hi @deboraelkin2 you raise some good points. The DSB is currently assessing OAS3 and ReDoc as well as refactoring the Swagger specification to be more modular. The view is to better provide a more manageable specification over time and address some of the points you raise. Also note that OAS3 has improvements to the API specification language features that are not available with Swagger 2.

Previously the DSB had consulted on the use of OAS 3 but at the time the feedback we received was that many API Gateways did not then have support for OAS 3. If that has changed and their is broader support it would make the move to OAS 3 more feasible.

Thanks for taking this request into consideration. I can't speak for other tools, but for what it's worth, the API Management platform from the company I represent (Google Cloud Apigee) certainly does support OAS 3.0.

CDR-API-Stream commented 4 years ago

Based on consideration of community feedback and review of issues and change requests, the focus of this iteration will address a number of themes:

perlboy commented 4 years ago

And just like that, Codegen is murdered: https://github.com/ConsumerDataStandardsAustralia/java-artefacts/issues/25#issuecomment-617575575

It seems the DSB is determined to make life as difficult as possible for everyone.

cdradr commented 4 years ago

What is or who is codegen? A member of open banking community who asked difficult questions?

perlboy commented 4 years ago

@cdradr https://github.com/ConsumerDataStandardsAustralia/standards/issues/108#issuecomment-615051753 CDS Codegen as referenced in my original comment.

CDR-API-Stream commented 4 years ago

Enhanced Error Handling Enhanced Error Handling has been broken down into component problem spaces for further consultation:

CDR-API-Stream commented 4 years ago

The final decision document has been reviewed and approved by the Data Standards Chair and it is included in the main issue above.