search

OpenAPITools / openapi-diff

Utility for comparing two OpenAPI specifications.
Apache License 2.0
803 stars 153 forks source link

Doesn't work: JsonParseException: Unrecognized token 'openapi': was expecting (JSON String, Number, Array, Object or token 'null', 'true' or 'false') #200

Open hasufell opened 3 years ago

hasufell commented 3 years ago

input files: https://gist.github.com/hasufell/1ae28db2326569d885c1e580515dc2f6

Full error:

$ docker run --rm -v /etc/passwd:/etc/passwd:ro -u (id -u):(id -g) -v (pwd):(pwd) -w (pwd) openapitools/openapi-diff specifications/api/swagger.yaml.v2020-11-03 specifications/api/swagger.yaml.v2020
-11-17-135-g5d316c5ef  --markdown api-diff.md
ERROR [io.swagger.parser.SwaggerCompatConverter] - failed to read resource listing
com.fasterxml.jackson.core.JsonParseException: Unrecognized token 'openapi': was expecting (JSON String, Number, Array, Object or token 'null', 'true' or 'false')
 at [Source: (StringReader); line: 1, column: 8]
    at com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1851)
    at com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:717)
    at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._reportInvalidToken(ReaderBasedJsonParser.java:2898)
    at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._handleOddValue(ReaderBasedJsonParser.java:1944)
    at com.fasterxml.jackson.core.json.ReaderBasedJsonParser.nextToken(ReaderBasedJsonParser.java:776)
    at com.fasterxml.jackson.databind.ObjectMapper._readTreeAndClose(ObjectMapper.java:4511)
    at com.fasterxml.jackson.databind.ObjectMapper.readTree(ObjectMapper.java:2940)
    at io.swagger.parser.SwaggerCompatConverter.readResourceListing(SwaggerCompatConverter.java:210)
    at io.swagger.parser.SwaggerCompatConverter.read(SwaggerCompatConverter.java:123)
    at io.swagger.parser.SwaggerCompatConverter.readWithInfo(SwaggerCompatConverter.java:94)
    at io.swagger.parser.SwaggerParser.readWithInfo(SwaggerParser.java:42)
    at io.swagger.v3.parser.converter.SwaggerConverter.readLocation(SwaggerConverter.java:89)
    at io.swagger.parser.OpenAPIParser.readLocation(OpenAPIParser.java:16)
    at org.openapitools.openapidiff.core.OpenApiCompare.readLocation(OpenApiCompare.java:116)
    at org.openapitools.openapidiff.core.OpenApiCompare.fromLocations(OpenApiCompare.java:90)
    at org.openapitools.openapidiff.core.OpenApiCompare.fromLocations(OpenApiCompare.java:77)
    at org.openapitools.openapidiff.cli.Main.main(Main.java:156)
ERROR [io.swagger.parser.SwaggerCompatConverter] - failed to read resource listing
com.fasterxml.jackson.core.JsonParseException: Unrecognized token 'openapi': was expecting (JSON String, Number, Array, Object or token 'null', 'true' or 'false')
 at [Source: (StringReader); line: 1, column: 8]
    at com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1851)
    at com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:717)
    at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._reportInvalidToken(ReaderBasedJsonParser.java:2898)
    at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._handleOddValue(ReaderBasedJsonParser.java:1944)
    at com.fasterxml.jackson.core.json.ReaderBasedJsonParser.nextToken(ReaderBasedJsonParser.java:776)
    at com.fasterxml.jackson.databind.ObjectMapper._readTreeAndClose(ObjectMapper.java:4511)
    at com.fasterxml.jackson.databind.ObjectMapper.readTree(ObjectMapper.java:2940)
    at io.swagger.parser.SwaggerCompatConverter.readResourceListing(SwaggerCompatConverter.java:210)
    at io.swagger.parser.SwaggerCompatConverter.read(SwaggerCompatConverter.java:123)
    at io.swagger.parser.SwaggerCompatConverter.readWithInfo(SwaggerCompatConverter.java:94)
    at io.swagger.parser.SwaggerParser.readWithInfo(SwaggerParser.java:42)
    at io.swagger.v3.parser.converter.SwaggerConverter.readLocation(SwaggerConverter.java:89)
    at io.swagger.parser.OpenAPIParser.readLocation(OpenAPIParser.java:16)
    at org.openapitools.openapidiff.core.OpenApiCompare.readLocation(OpenApiCompare.java:116)
    at org.openapitools.openapidiff.core.OpenApiCompare.fromLocations(OpenApiCompare.java:90)
    at org.openapitools.openapidiff.core.OpenApiCompare.fromLocations(OpenApiCompare.java:77)
    at org.openapitools.openapidiff.cli.Main.main(Main.java:156)
Unexpected exception. Reason: Cannot read old OpenAPI spec
java.lang.RuntimeException: Cannot read old OpenAPI spec
    at org.openapitools.openapidiff.core.OpenApiCompare.notNull(OpenApiCompare.java:106)
    at org.openapitools.openapidiff.core.OpenApiCompare.fromSpecifications(OpenApiCompare.java:101)
    at org.openapitools.openapidiff.core.OpenApiCompare.fromLocations(OpenApiCompare.java:90)
    at org.openapitools.openapidiff.core.OpenApiCompare.fromLocations(OpenApiCompare.java:77)
    at org.openapitools.openapidiff.cli.Main.main(Main.java:156)
joschi commented 3 years ago

@hasufell Thanks for reporting this!

I think the reason for this rather unhelpful error message is the use of merge keys (<<: ...) in the OpenAPI specification you've linked to.

YAML merge keys are not very well specified and not very well supported by Jackson (jackson-dataformat-yaml).

Multiple sequential merge keys on the same level are generally a problem for this stack.

In fact, YAML 1.2 removes them altogether, so that you ideally shouldn't rely on them.

Here's the spec and the diff to your original spec I've been using: spec.yaml spec.diff

You should be able to get a more detailed error description by setting the log level of openapi-diff to INFO or WARN:

openapi-diff -l INFO old.yaml new.yaml

FWIW, OpenAPI 3.0.3 recommends using YAML 1.2 (which doesn't support merge keys): https://swagger.io/specification/#format

In order to preserve the ability to round-trip between YAML and JSON formats, YAML version 1.2 is RECOMMENDED along with some additional constraints:

  • Tags MUST be limited to those allowed by the JSON Schema ruleset.
  • Keys used in YAML maps MUST be limited to a scalar string, as defined by the YAML Failsafe schema ruleset.
hasufell commented 3 years ago

I applied the patch and now I get:

docker run --rm -v /etc/passwd:/etc/passwd:ro -u (id -u):(id -g) -v (pwd):(pwd) -w (pwd) openapitools/openapi-diff specifications/api/swagger.yaml.v2020-11-1
7-135-g5d316c5ef specifications/api/swagger.new --markdown api-diff.md
Unexpected exception. Reason: invalid oneOf schema
java.lang.IllegalArgumentException: invalid oneOf schema
    at org.openapitools.openapidiff.core.compare.schemadiffresult.ComposedSchemaDiffResult.getMapping(ComposedSchemaDiffResult.java:103)
    at org.openapitools.openapidiff.core.compare.schemadiffresult.ComposedSchemaDiffResult.diff(ComposedSchemaDiffResult.java:61)
    at org.openapitools.openapidiff.core.compare.SchemaDiff.computeDiff(SchemaDiff.java:316)
    at org.openapitools.openapidiff.core.compare.SchemaDiff.computeDiff(SchemaDiff.java:27)
    at org.openapitools.openapidiff.core.compare.ReferenceDiffCache.cachedDiff(ReferenceDiffCache.java:51)
    at org.openapitools.openapidiff.core.compare.SchemaDiff.diff(SchemaDiff.java:282)
    at org.openapitools.openapidiff.core.compare.ContentDiff.diff(ContentDiff.java:33)
    at org.openapitools.openapidiff.core.compare.RequestBodyDiff.computeDiff(RequestBodyDiff.java:77)
    at org.openapitools.openapidiff.core.compare.RequestBodyDiff.computeDiff(RequestBodyDiff.java:17)
    at org.openapitools.openapidiff.core.compare.ReferenceDiffCache.cachedDiff(ReferenceDiffCache.java:51)
    at org.openapitools.openapidiff.core.compare.RequestBodyDiff.diff(RequestBodyDiff.java:34)
    at org.openapitools.openapidiff.core.compare.OperationDiff.diff(OperationDiff.java:42)
    at org.openapitools.openapidiff.core.compare.PathDiff.diff(PathDiff.java:35)
    at org.openapitools.openapidiff.core.compare.PathsDiff.lambda$diff$2(PathsDiff.java:67)
    at java.util.LinkedHashMap$LinkedKeySet.forEach(LinkedHashMap.java:559)
    at org.openapitools.openapidiff.core.compare.PathsDiff.diff(PathsDiff.java:40)
    at org.openapitools.openapidiff.core.compare.OpenApiDiff.compare(OpenApiDiff.java:95)
    at org.openapitools.openapidiff.core.compare.OpenApiDiff.compare(OpenApiDiff.java:66)
    at org.openapitools.openapidiff.core.OpenApiCompare.fromSpecifications(OpenApiCompare.java:101)
    at org.openapitools.openapidiff.core.OpenApiCompare.fromLocations(OpenApiCompare.java:90)
    at org.openapitools.openapidiff.core.OpenApiCompare.fromLocations(OpenApiCompare.java:77)
    at org.openapitools.openapidiff.cli.Main.main(Main.java:156)

Note that I have proof that the schema is entirely correct:

$ openapi-spec-validator --schema 3.0.0 specifications/api/swagger.new                                                                       1018ms  Mon 09:59
OK
Yayan89 commented 3 years ago

We are also experiencing the same error with version 5.1.0

plugins {
    id "org.openapi.generator" version "5.1.0"
}
com.fasterxml.jackson.core.JsonParseException: Unrecognized token 'openapi': was expecting (JSON String, Number, Array, Object or token 'null', 'true' or 'false')
 at [Source: (String)"openapi: 3.0.3
info:

in our yml, we don't have "<<:".

https://gist.github.com/Yayan89/099cbf29427f24ffd84484c39710a593

joschi commented 3 years ago

@Yayan89 I'm unable to reproduce your issue with OpenAPI-Diff 2.0.0-beta.8 and your OpenAPI specification:

--- ourContract.yml 2021-03-30 16:08:12.000000000 +0200
+++ ourContract2.yml    2021-03-30 16:07:00.000000000 +0200
@@ -3,7 +3,7 @@
   title: "Partage de documents avec l'externe"
   description: |
     API pour partager des documents.
-  version: "1.0.0"
+  version: "2.0.0"

 servers:
   - url: /api/document-numerique/gerer-partage/pde/v1
@@ -588,13 +588,6 @@
           type: string
           description: La date où le fichier sera automatiquement envoyé à la corbeille.
           format: date-time
-        dateSuppression:
-          type: string
-          description: La date où le fichier sera automatiquement supprimé du système.
-          format: date-time
-        taille:
-          type: string
-          description: Taille en bytes

     MiseAJourDossier:
       type: object
# docker run --rm -t -v $(pwd):/specs:ro openapitools/openapi-diff:2.0.0-beta.8 /specs/ourContract.yml /specs/ourContract2.yml
==========================================================================
==                            API CHANGE LOG                            ==
==========================================================================
                   Partage de documents avec l'externe
--------------------------------------------------------------------------
--                            What's Changed                            --
--------------------------------------------------------------------------
- GET    /dossiers/{idDossier}/elements
  Return Type:
    - Changed 200 OK
      Media types:
        - Changed application/json
          Schema: Broken compatibility
          Missing property: fichiers[n].dateSuppression (string)
          Missing property: fichiers[n].taille (string)
- POST   /fichiers
  Return Type:
    - Changed 201 Created
      Media types:
        - Changed application/json
          Schema: Broken compatibility
          Missing property: dateSuppression (string)
          Missing property: taille (string)
- GET    /fichiers/{idFichier}
  Return Type:
    - Changed 200 OK
      Media types:
        - Changed application/json
          Schema: Broken compatibility
          Missing property: dateSuppression (string)
          Missing property: taille (string)
- PATCH  /fichiers/{idFichier}
  Return Type:
    - Changed 200 OK
      Media types:
        - Changed application/json
          Schema: Broken compatibility
          Missing property: dateSuppression (string)
          Missing property: taille (string)
--------------------------------------------------------------------------
--                                Result                                --
--------------------------------------------------------------------------
                 API changes broke backward compatibility
--------------------------------------------------------------------------
harsimranmaan commented 3 years ago

I ran into the same failure. It was caused due to the usage of a potentially unsupported version of openapi. I tried 3.1.0 in https://github.com/bigpanther/trober/runs/2284521332?check_suite_focus=true but it did not work. 3.0.3 works fine.

atigm commented 3 years ago

I have the same probleme with 3.1.x, but 3.0.x works fine.

hafizali05 commented 3 years ago

anyone has made this work ? :-|

pdkgamage commented 3 years ago

anyone has solution for this?

petkostas commented 2 years ago

Any updates about 3.1.x support?

joschi commented 2 years ago

@petkostas You can track and contribute to the respective issue in the Swagger Parser project which is being used by OpenAPI Diff for parsing OpenAPI specifications:

https://github.com/swagger-api/swagger-parser/issues/1535

AntonTevs commented 1 year ago

The solution is Do not put SPACES in the path to your project and yaml. I hope it can help you.

RafaelSimeoni commented 11 months ago

@AntonTevs Thank you!!