search

stoplightio / spectral

A flexible JSON/YAML linter for creating automated style guides, with baked in support for OpenAPI (v3.1, v3.0, and v2.0), Arazzo v1.0, as well as AsyncAPI v2.x.
https://stoplight.io/spectral
Apache License 2.0
2.48k stars 233 forks source link

Invalid JSON + invalid OAS is not recognized as such #2641

Open bereg2k opened 3 months ago

bereg2k commented 3 months ago

Discussed in https://github.com/stoplightio/spectral/discussions/2630

Originally posted by **bereg2k** June 2, 2024 Hello! I have an **invalid JSON** which is **invalid OAS** too, but linting it doesn't report any errors: `invalid_spec.json` ```json { "openap": "3.0.3", "info": { "title": "", "version": "" }, } ``` - there's trailing `,` without any other members (invalid JSON) - `"openap"` instead of `"openapi"` (invalid OAS) - no `"paths"` (invalid OAS). My ruleset is... `.spectral.yaml` ```yaml extends: [[spectral:oas, all]] ``` The output is: ```bash % spectral lint path/to/file/invalid_spec.json -v -D Found 52 rules (52 enabled) Linting /path/to/file/invalid_spec.json No results with a severity of 'error' found! ``` What am I doing wrong here?
daniel-white commented 3 months ago

just a few thoughts

  1. the trailing comma problem isn't really the domain of spectral - spectral cares more about the sematics of the content than the syntax. another tool might help!
  2. im thinking like a "format/detect/core property" configuration for rule sets that the core engine would consult to ensure that one of those must exist: json schema, oas, asyncapi, etc. it might have to be behind a new flag but if the "detector" doesn't find one, then its an error
  3. since the ruleset didn't see this as an oas file, then it didnt run, hence missing paths wasn't checked
bereg2k commented 3 months ago

@daniel-white Thanks for your response!

  1. Can you please recommend me something appropriate for the syntax validations though?
  2. Should I create a separate issue for it? ("openap" key issue)
  3. But shouldn't the tool see all the files I am trying to validate as OAS files explicitly, and throw errors accordingly? The file could be incorrect due to human error, and internally Spectral "decides" that it's not OAS and so won't do any validation... That's kinda weird (if I understood this correctly).
mheguy commented 2 months ago

internally Spectral "decides" that it's not OAS

It's just code. If there is an "openapi" key in the root then it applies oas rules.

As Daniel mentioned, the tool should probably say something if it doesn't match on oas, aas, or json schema. Outside of that, there's not much to do.

ponelat commented 1 month ago

@mnaumanali94 do we have any use-cases for "passing through unknown spec files as error free"? I would consider throwing an error if spectral can't identify the specification?

Couple of ways we can improve this: