Open tedepstein opened 6 years ago
@darrelmiller, please have a look & see if this captures the ideas we discussed on the call today.
BTW:
@tedepstein Looks great to me. I'm going to add a note to tomorrow's agenda to discuss what next steps we can take. If you are able to make the meeting then it would be great to hear your thoughts and how we can progress.
The Open Geospatial Consortium (OGC) runs a compliance test program at http://cite.opengeospatial.org/. This program is supported by the Team Engine test server at https://github.com/opengeospatial/teamengine. Team Engine executes test scripts written in TestNG (http://testng.org/doc/index.html). As part of their Testbed 14 initiative, the OGC will develop compliance tests for their OpenAPI based WFS 3.0 spec. This work may provide us with most of what we need for a general purpose OpenAPI compliance test. Any interest?
@darrelmiller and @cmheazel, sorry to have missed your earlier comments. My Gmail filter was a bit overzealous, and I just corrected that.
I'll be on Monday's call to discuss next steps, and I'll take a look at what the OGC has done. Thanks!
My live notes from Monday's TSC call:
OAS3 Certification Testing 2018-04-30.pptx
Text content, for SEO & inline reading:
End State: Certification Process Overview
Start with OpenAPI 3.0 Document “Consumers”
Questions, Challenges…
Rules and Test Cases for Validation
Existing Validators
Discussed on TSC Meeting: April 2, 2018:
Rules and test cases in a conformance test suite might have a descriptive structure, something like this:
OpenAPI implementations, including editors, generators, documentation formats, etc., are expected to validate the content, correctly detect error and warning conditions (without false-positives on valid content), expose and handle errors and warnings in whatever way is appropriate for that implementation.
Some goals, and possibly non-goals, pending discussion:
Consider all of the above as a straw-man proposal, for discussion.