search

stoplightio / spectral-action

GitHub Action wrapper for Spectral - a JSON/YAML/OpenAPI/AsyncAPI/etc linter with custom rule support.
https://stoplight.io/open-source/spectral
Apache License 2.0
90 stars 46 forks source link

Error/warning messages are added to a completely different job #646

Open tristan957 opened 2 years ago

tristan957 commented 2 years ago

Context

Recently I have been working on this PR: https://github.com/hse-project/hse/pull/316. During my work, I have created an OpenAPI document at https://github.com/hse-project/hse/blob/tristan957/rest/docs/openapi.yaml. I thought I would add some linting for the file, and this action was the best one I could find (nice job by the way, really useful, also installed the VSCode extension). The GitHub Action workflow is defined at https://github.com/hse-project/hse/blob/tristan957/rest/.github/workflows/openapi.yaml. You can see the workflow's name is "OpenAPI" and the singular job's name is "lint". During the course of my development, I have discovered this interesting behavior from this action.

Here is my spectral config file: https://github.com/hse-project/hse/blob/tristan957/rest/.spectral.yaml

Current Behavior

What is currently happening is that the messages that this action generates get added to a job that doesn't exist.

image

Note the uppercase "Lint". In my multiple revisions of this PR, I have also seen this job get added to other workflows that aren't the "OpenAPI" workflow described above.

https://github.com/hse-project/hse/runs/7460788923

Note that there are currently no issues with the file, so no messages were added.

Edit: image

Expected Behavior

I would have expected the messages to be included in the job that actually generates them.

Steps to Reproduce

  1. Create workflow to run spectral
  2. Run the workflow
  3. Denote job that was created that hasn't previously existed

This seems partially related to #506. At least that also mentions a "Lint" job.

tristan957 commented 2 years ago

I'm pretty sure the issue is that the extension is creating a check and not updating a check after reading the source code.

tristan957 commented 2 years ago

Ok. Since I created this issue, I have become a little bit more knowledgeable about how checks like this one are supposed to work.

I recently integrated CodeQL into our CI using github/codeql-action. The CodeQL workflow that I wrote runs green all the time regardless of whether CodeQL actually found any issues or now. The workflow would fail if it couldn't download the docker image however or some other infrastructure issue like that.

The codeql-action then creates a new check like this:

image

And depending on whether the CodeQL analysis turned anything up, it'll be red or green with the annotations attached to that check. Note that I didn't create this workflow or job.

t-huyeng commented 2 years ago

@tristan957 do you have any update on this? I am experiencing the same. Sometimes the annotations just do not get added at all.

For example here: https://github.com/t-huyeng/hlnug-api/actions/runs/3356154510

Sometimes twice: https://github.com/t-huyeng/api-doc-template/actions/runs/3306486009

tristan957 commented 2 years ago

No updates. I just kind of live with this broken action for now...

t-huyeng commented 2 years ago

Maybe @kaylachun (most recent pull-request) or @P0lip can help? Is this action still maintained by Stoplight?

tristan957 commented 1 year ago

Seems like there has been action on this repo recently. Perhaps someone will see this issue.