I can't seem to get my Github Action to fail...

[mattgrande]

I'm trying to get my Github build to fail if my Accessibility Score falls below a certain threshold.

I suspect it's because it's not reading my lighthouserc.json file correctly (or, more likely, my file is incorrect).

Please tell me what I'm doing wrong! (Also, please tell me if there's somewhere better to ask questions like this)

My lighthouse.yml

name: Lighthouse

on: [push]

jobs:
  lighthouse:
    # Setup cut for berevity

    - name: Audit URLs using Lighthouse
      uses: treosh/lighthouse-ci-action@v7
      with:
        urls: |
          http://[my-url]/
          http://[my-url]/welcome
        configPath: ./.github/lighthouse/lighthouserc.json
        uploadArtifacts: true
        temporaryPublicStorage: true

And the config file (lighthouserc.json)

{
  "ci": {
    "assert": {
      "assertions": {
        "color-contrast": "off",
        // NOTE: I've tried both 0.99 and 99, and neither cause my build to fail
        "categories:accessibility": ["error", {"minScore": 99}],
        "categories:best-practices": "off",
        "categories:performance": "off",
        "categories:seo": "off"
      }
    }
  }
}

Even though everything's mostly off, all the assertions still get run, and even though my required A11y score is 99 (and the reported score is 92), my build still passes.

Any insight here would be greatly appreciated!

[alekseykulikov]

Hey, it seems there's a misconfiguration, but I can't see where. I made a quick failing example similar to your config: https://github.com/treosh/lighthouse-ci-action/runs/1935692999?check_suite_focus=true Let us know what the problem was.