search

placekit / api-reference

PlaceKit API reference
https://api.placekit.io
MIT License
7 stars 0 forks source link

Can't parse openapi.yml as valid YAML #3

Open MikeRalphson opened 1 year ago

MikeRalphson commented 1 year ago 
boast https://raw.githubusercontent.com/placekit/api-reference/main/openapi.yml
GET https://raw.githubusercontent.com/placekit/api-reference/main/openapi.yml
YAMLSyntaxError: All collection items must start at the same column at line 1, column 1:

curl --location --request POST 'https://api.placekit.co/search' \
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^…

    at Collection.parse (/Users/mikeralphson/node/oas-kit/packages/oas-resolver/node_modules/yaml/dist/parse-cst.js:331:24)
    at /Users/mikeralphson/node/oas-kit/packages/oas-resolver/node_modules/yaml/dist/parse-cst.js:1605:29
    at Document.parseContents (/Users/mikeralphson/node/oas-kit/packages/oas-resolver/node_modules/yaml/dist/parse-cst.js:622:26)
    at Document.parse (/Users/mikeralphson/node/oas-kit/packages/oas-resolver/node_modules/yaml/dist/parse-cst.js:685:19)
    at Object.parse (/Users/mikeralphson/node/oas-kit/packages/oas-resolver/node_modules/yaml/dist/parse-cst.js:1723:18)
    at parseDocument (/Users/mikeralphson/node/oas-kit/packages/oas-resolver/node_modules/yaml/dist/index.js:43:24)
    at Object.parse (/Users/mikeralphson/node/oas-kit/packages/oas-resolver/node_modules/yaml/dist/index.js:55:15)
    at mainPass (/Users/mikeralphson/node/oas-kit/packages/oas-resolver/index.js:171:21)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Object.resolve (/Users/mikeralphson/node/oas-kit/packages/oas-resolver/index.js:239:3) {
  nodeType: 'MAP',
  range: Range { start: 0, end: 390 },
  linePos: { start: { line: 1, col: 1 }, end: { line: 11, col: 7 } }
}
All collection items must start at the same column at line 1, column 1:

curl --location --request POST 'https://api.placekit.co/search' \
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^…
noclat commented 1 year ago

Hi @MikeRalphson, could you share a codesnippet? I've tried to first check with online validators and it seems ok, redocly linter is all green too, and I've tried to parse it with YAML.parse() from the yaml node package from both local file and distant from raw.githubusercontent.com and get no error so far.

MikeRalphson commented 1 year ago

It's in one of the included code-snippet files...

boast --verbose --verbose https://raw.githubusercontent.com/placekit/api-reference/main/openapi.yml
GET https://raw.githubusercontent.com/placekit/api-reference/main/openapi.yml
Resolving a Symbol(docType-openapi) parent none at https://raw.githubusercontent.com/placekit/api-reference/main/openapi.yml Previous []
GET https://raw.githubusercontent.com/placekit/api-reference/main/code_samples/curl/search.sh Previous [
  'https://raw.githubusercontent.com/placekit/api-reference/main/openapi.yml'
]
GET https://raw.githubusercontent.com/placekit/api-reference/main/code_samples/go/search.go Previous [
  'https://raw.githubusercontent.com/placekit/api-reference/main/openapi.yml'
]
GET https://raw.githubusercontent.com/placekit/api-reference/main/code_samples/js/search.js Previous [
  'https://raw.githubusercontent.com/placekit/api-reference/main/openapi.yml'
]
GET https://raw.githubusercontent.com/placekit/api-reference/main/code_samples/python/search.py Previous [
  'https://raw.githubusercontent.com/placekit/api-reference/main/openapi.yml'
]
GET https://raw.githubusercontent.com/placekit/api-reference/main/code_samples/ruby/search.rb Previous [
  'https://raw.githubusercontent.com/placekit/api-reference/main/openapi.yml'
]
GET https://raw.githubusercontent.com/placekit/api-reference/main/code_samples/curl/reverse.sh Previous [
  'https://raw.githubusercontent.com/placekit/api-reference/main/openapi.yml'
]
GET https://raw.githubusercontent.com/placekit/api-reference/main/code_samples/go/reverse.go Previous [
  'https://raw.githubusercontent.com/placekit/api-reference/main/openapi.yml'
]
GET https://raw.githubusercontent.com/placekit/api-reference/main/code_samples/js/reverse.js Previous [
  'https://raw.githubusercontent.com/placekit/api-reference/main/openapi.yml'
]
GET https://raw.githubusercontent.com/placekit/api-reference/main/code_samples/python/reverse.py Previous [
  'https://raw.githubusercontent.com/placekit/api-reference/main/openapi.yml'
]
GET https://raw.githubusercontent.com/placekit/api-reference/main/code_samples/ruby/reverse.rb Previous [
  'https://raw.githubusercontent.com/placekit/api-reference/main/openapi.yml'
]
YAMLSyntaxError: All collection items must start at the same column at line 1, column 1:

curl --location --request POST 'https://api.placekit.co/search' \
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^…

Ok - the problem is using $ref within an extension is not guaranteed to work, it's at best undefined behaviour, at worst $ref is just another property and doesn't get resolved. You're using plain text (markdown) files as the target of $refs too, rather than the more normal JSON or YAML (things with defined semantics in the OpenAPI world).

ReDoc is very lenient here, but it doesn't necessarily help people write interoperable OpenAPI definitions (cc: @lornajane)

lornajane commented 1 year ago

Thanks for the ping, this is interesting! Redocly tooling will usually expand a $ref in a context where it's clear what the user intended - but the OpenAPI specification actually only technically supports it in a limited set of locations. This usually works out pretty well for rendering docs, but I can see that it could confuse other tools that trip up on the unexpected fields. (see also: docs for the x-codeSample support in Redocly https://redocly.com/docs/api-reference-docs/specification-extensions/x-code-samples/) I wonder if we could add a linting rule to catch this sort of thing for situations where there are other tools in the chain that need a stricter interpretation of the spec.

MikeRalphson commented 1 year ago

That sounds excellent, thanks for pitching in on this issue!

noclat commented 1 year ago

@MikeRalphson we've investigated a bit further, and we noticed a few things:

npx oas-validate -v --validateSchema=first --lint openapi.yml
Gathering...
openapi.yml
  PlaceKit API Reference 1.0.0
  https://api.placekit.co 

Tests: 1 passing, 0 failing, 0 warnings

Anything else we can do on our side?

MikeRalphson commented 1 year ago

Thanks. BTW boast isn't unmaintained, the last commit to it was within the last 10 days. It just needs a new release.

lornajane commented 1 year ago

To close the loop on this, we've added a rule to check for the more restricted/correct set of $ref usage. I'd be interested to know if this helps catch the problem, so feedback is welcome! Docs: https://redocly.com/docs/cli/rules/spec-strict-refs/

raphi commented 1 year ago

Hi @lornajane , thanks for following up! I've added the new rule and the linter passed successfully.

Taking a closer look at the issue and as @MikeRalphson said in comments, the problem is referencing a non-yaml file for the sample's source. Redocly extract and display the code sample correctly, but the syntax is not strictly valid. (BTW there is no example in the documentation about referencing a code sample file, just a simple inline example. Any idea how to do it properly @lornajane ?)

I've changed the samples to be valid yaml reference files and boast validation is now passing (#4). It's not ideal because code samples aren't in there original files extension (same source files used for formatting, linting and testing) but at least both boast and redocly now pass!

lornajane commented 1 year ago

I did some digging on this, and found an example in the Rebilly API (Redocly and Rebilly are sister companies, lots of Redocly functionality serves Rebilly's needs!) https://github.com/Rebilly/api-definitions/blob/main/openapi/paths/customers%40%7Bid%7D.yaml#L32 It probably won't validate in any other tooling (perhaps that's less of a problem when it's inside an x- extension), but it does mean that the code can be in its own file format, and be linted or even executed there to test it, which is why we did it this way.

I'm not sure why it's not in the docs but I'll add it!

raphi commented 1 year ago

Thanks for the many examples!

I ran Mike's validator on your specs and indeed the $ref targeting non-yaml/json files is invalid: 

$ npx boast https://raw.githubusercontent.com/Rebilly/api-definitions/main/openapi/openapi.yaml
[...]
GET https://raw.githubusercontent.com/Rebilly/api-definitions/main/openapi/code_samples/PHP-SDK-3.0/customers/get.php 
YAMLSyntaxError: All collection items must start at the same column at line 1, column 1:

$service = new Rebilly\Sdk\CoreService($client);
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^…
[...]

@MikeRalphson anything else I can do on my side to pass APIGuru's validation?

raphi commented 10 months ago

@MikeRalphson up?