search

microsoft / OpenAPI.NET

The OpenAPI.NET SDK contains a useful object model for OpenAPI documents in .NET along with common serializers to extract raw OpenAPI JSON and YAML documents from the model.
MIT License
1.41k stars 232 forks source link

Azure API Management schema validator for CI jobs #529

Open evertonmc opened 4 years ago

evertonmc commented 4 years ago

Please provide a documentation on how to validate openapi and swagger documents for the Azure APIM import. I can't find a way to get the same errors in autorest (e.g.) thrown by the azure api management import. Also tried a lot of other validators, and all of them seem to consider some specs as valid where apim would reject with schema validation error.

autorest

autorest --v3 --azure-validator --input-file=openapi.json

AutoRest code generation utility [cli version: 3.0.6247; node: v12.19.0, max-memory: 2048 gb]
(C) 2018 Microsoft Corporation.
https://aka.ms/autorest
   Loading AutoRest core      '/home/ema/.autorest/@autorest_core@3.0.6320/node_modules/@autorest/core/dist' (3.0.6320)
   Installing AutoRest extension '@microsoft.azure/classic-openapi-validator' (~1.0.9)
   Installed AutoRest extension '@microsoft.azure/classic-openapi-validator' (~1.0.9->1.0.14)
   Installing AutoRest extension '@microsoft.azure/openapi-validator' (~1.0.2)
   Installed AutoRest extension '@microsoft.azure/openapi-validator' (~1.0.2->1.0.4)
[21.56 s] Generation Complete

azure portal import

Parsing error(s): The key 'KeyValuePair`2' in 'schemas' of components MUST match the regular expression '^[a-zA-Z0-9\.\-_]+$'. [#/components]
Parsing error(s): The input OpenAPI file is not valid for the OpenAPI specification https://github.com/OAI/OpenAPI-Specification/blob/master/versions/3.0.1.md (schema https://github.com/OAI/OpenAPI-Specification/blob/master/schemas/v3.0/schema.yaml).

This is very important for CI workflows to use the validation as PR gate before accepting merges.

darrelmiller commented 3 years ago

AutoREST only validates a small number of constraints from the OpenAPI spec. The OpenAPI.NET library validates all constraints that are MUST from the specification. The specific rule that is being violated is here http://spec.openapis.org/oas/v3.0.3#fixed-fields-5 I expect the reason why you are seeing the error above is due to the fact that there is a component schema name with an invalid character. I have seen this happen when people use parentheses in schema names.

The one area that I consider a false positive error is when example values are identified as having the wrong time. I need to remove this as a default validator. If you see errors about issues that are not explicitly prevented by the OpenAPI specification, I will definitely remove those checks.

darrelmiller commented 3 years ago

I have been working on a command line tool that will allow validation. Would that help you to pre-validate OpenAPI documents?

evertonmc commented 3 years ago

@darrelmiller that's exactly what would help here. The errors are ok, but I wan't to get them before I try to upload the specs to the api management. A CLI tool with the same validation logic would make that happen.

Blackbaud-ChristiSchneider commented 3 years ago

IBM has a command line validation tool - https://github.com/IBM/openapi-validator It won't have identical logic to the APIM errors but I've been using the same logic to validate our documents prior to upload.

darrelmiller commented 3 years ago

@Blackbaud-ChristiSchneider Yeah, in theory any validator that properly validates the spec would be equivalent, but there are definitely some grey areas when it comes to validation. This tool currently is overly aggressive in trying to validate data types of example data which causes all kinds of problems.

evertonmc commented 3 years ago

I have been working on a command line tool that will allow validation. Would that help you to pre-validate OpenAPI documents?

@darrelmiller can you share this tool a.t.m.?

darrelmiller commented 3 years ago

It is here https://github.com/microsoft/OpenAPI.NET/tree/vnext/src/Microsoft.OpenApi.Tool