Ability to skip some assertions only for several URLs

[pahan35]

We have several pages on our resource where we use outer HTML where are some problems with some lighthouse assertions.

Is there any possibility to skip some checks only for specific URLs?

Or maybe it'll be even better to have a possibility to skip assertions for the whole container with third-party markup by adding something like specific data attribute to container like data-lighthouse-skip-assertions.

[patrickhulce]

Yep, see the docs

Or maybe it'll be even better to have a possibility to skip assertions for the whole container with third-party markup by adding something like specific data attribute to container like data-lighthouse-skip-assertions.

You'd need to take this up with Lighthouse core repo, though I can say I think it's highly unlikely to be pursued.

[pahan35]

Unfortunately, assertMatrix doesn't work well for me: I have a big disabled assertions list and looks like I need to duplicate it for each separate url pattern to be able to use assertMatrix

  "assert": {
      "preset": "lighthouse:no-pwa",
      "assertions": {
        "bootup-time": "off",
        "dom-size": "off",
        "efficient-animated-content": "off",
        "first-contentful-paint": "off",
        "first-cpu-idle": "off",
        "first-meaningful-paint": "off",
        "font-display": "off",
        "interactive": "off",
        "mainthread-work-breakdown": "off",
        "max-potential-fid": "off",
        "offscreen-images": "off",
        "redirects": "off",
        "redirects-http": "off",
        "render-blocking-resources": "off",
        "speed-index": "off",
        "time-to-first-byte": "off",
        "total-byte-weight": "off",
        "unminified-css": "off",
        "unminified-javascript": "off",
        "unused-css-rules": "off",
        "uses-http2": "off",
        "uses-long-cache-ttl": "off",
        "uses-optimized-images": "off",
        "uses-rel-preconnect": "off",
        "uses-responsive-images": "off",
        "uses-rel-preload": "off",
        "uses-webp-images": "off",
        "uses-text-compression": "off",
        "duplicate-id": "off",
        "errors-in-console": "off",
        "external-anchors-use-rel-noopener": "off",
        "link-text": "off"
      },
      "assertMatrix": [
        {
          "matchingUrlPattern": "/some/url",
          "assertions": {
            "list": "off"
          }
        },
        {
          "matchingUrlPattern": "/another/url",
          "assertions": {
            "button-name": "off",
          }
        }
      ]
    }

Are you going to add some possibility to proceed with such cases?

Also, as you see, I have disabled all assertions from performance because I disabled this category on collect. I guess it would be better to add a possibility to skip assertions for disabled categories automatically.

    "collect": {
      "settings": {
        "onlyCategories": ["accessibility", "best-practices", "seo"]
      },
    }

[patrickhulce]

I have a big disabled assertions list and looks like I need to duplicate it for each separate url pattern

Could you use a lighthouserc.js instead of JSON and put the list into a variable?

I guess it would be better to add a possibility to skip assertions for disabled categories automatically.

I'm a little concerned about the automagicness of inferring other command configuration via the deeply nested properties of other commands. Additionally, we don't really want to encourage folks to disable collection of categories since it makes diff'ing and debugging much harder and less useful (especially performance which contains all of the diagnostic audits, screenshots, etc). It's fine if someone wants to go through the effort to customize the run and know what they're doing but introducing functionality and options to make it easy isn't the interests of the project.

Would a reasonable alternative to your goals be...

1. More presets for each category lighthouse:accessibility, lighthouse:seo, etc
2. The ability to specify multiple presets

to turn your config into preset: ['lighthouse:accessibility', 'lighthouse:seo', 'lighthouse:best-practices']?

[klarkc]

I found a workaround to do this with yaml using anchors in a less verbose way, here is an example:

---
ci:
  assert:
    assertMatrix:
    - matchingUrlPattern: ^https?:\/\/.*\/(?!checkout|store).*
      preset: &preset lighthouse:recommended
      assertions: &assertions
        categories:performance:
        - error
        - minScore: 0.25
        categories:seo:
        - error
        - minScore: 0.7
        categories:accessibility:
        - error
        - minScore: 0.7
        categories:best-practices:
        - error
        - minScore: 0.7
        categories:pwa:
        - error
        - minScore: 0.7
    - matchingUrlPattern: /checkout
      preset: *preset
      assertions:
        <<: *assertions
        non-composited-animations: off
    - matchingUrlPattern: /store
      preset: *preset
      assertions:
        <<: *assertions
        is-crawlable: off

Keep in mind that the first matchingUrlPattern is matching any URLs except the overrides.