Default / Built-in ruleset behavior doesn't work as described

bradlet commented 2 years ago

When trying to specify a core/provided ruleset, we did so before by specifying spectral-ruleset: "spectral:oas3". This stopped working because the ruleset is not being found in the github workspace (Separate issue potentially). Supposedly spectral-ruleset is an optional parameter, and if you don't use it, a built-in ruleset should be used. But getDefaultRulesetFile only searches the CWD for a default ruleset. If none are found, it errors out -- as opposed to using a built-in / core ruleset.

Context

Extra lint job, which pops up after job that contains the spectral action lint step, never ends:

This one's been going for like 3 days now.

Current Behavior

Aside from the extra lint job that never ends, the lint step in our build job ends in error, but is considered a success

When not specifying any spectral-ruleset you see:

When specifying a spectral-ruleset, attempting to use some existing core ruleset, you see this (Note: tried 'spectral:oas' in this run, but in others tried 'spectral:oas3' and 'oas3' -- same result each time)

Expected Behavior

1) When specifying some predefined spectral ruleset such as "spectral:oas3" it works as it did before (we had been on v0.5.5 for at least a year) and the openApiV3 ruleset is used. 2) When the optional ruleset parameter isn't included, some built-in ruleset is used as described in the README.

Possible Workaround/Solution

Make getDefaultRulesetFile look for built-in rulesets if no ruleset is provided
Make "spectral_ruleset" param values prefixed with 'spectral:' search stoplight-core (or stoplight-rulesets? However that works/used to work) for a matching ruleset with the suffixed tag.

Steps to Reproduce

Setup and run a job that looks like this:

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v1
      - name: Spectral Lint
        uses: stoplightio/spectral-action@v0.5.5
        with:
          file_glob: ./public/api/*.yaml
          spectral_ruleset: "spectral:oas3"
          repo_token: ${{ secrets.GITHUB_TOKEN }}
.
.
.

Or like this:

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Spectral Lint
        uses: stoplightio/spectral-action@v0.8.1
        with:
          file_glob: ./public/api/*.yaml
          repo_token: ${{ secrets.GITHUB_TOKEN }}
.
.
.

Environment

Version used: v0.8.2
Environment name and version (e.g. Chrome 39, node.js 5.4): ubuntu-latest
Operating System and version (desktop or mobile): desktop
Link to your environment/workspace/project: private project

bradlet commented 2 years ago

Oh also, unrelated, the 'spectral-ruleset' link in the README 404's now. IDK what page that used to point to, but just calling it out here in-case someone who knows what that should point to sees this 🤷

bradlet commented 2 years ago

Update: My suspicion is that, back in November of 2019, when upgrading from v0.5.5 -> v0.6~, the 'fallback to built-in ruleset' behavior was broken. That update is what added support for custom rulesets. I surmise that this snuck by because most people just went ahead and added a local yml spec for their ruleset.

Suggested fix(es) now can simply be to update the README so that it doesn't indicate that there is any fallback behavior to use a core ruleset. Can also add a note on adding a .spectral.yaml w/ only the contents: extends: ["spectral:oas"]

in order to use a core ruleset w/o any configuration...

mnaumanali94 commented 2 years ago

@bradlet The readme has been updated. You were right with your suspicion. :D

aharpervc commented 1 year ago

I found this issue when setting up the GH action, and it didn't work even when I followed the readme instructions...

Can also add a note on adding a .spectral.yaml w/ only the contents: extends: ["spectral:oas"]

This is correct. However, the readme says:

if you'd like to simply use a core ruleset without additional configuration, create a .spectral.yml in you repository's root with only the contents: extends: ["spectral:{rulesetTagHere}"]

This is incorrect. I recommend adjusting the readme to what @bradlet suggested, literally

stoplightio / spectral-action