Flaky web_component_test action

[jcscottiii]

The web_component_test CI action is a little flaky. An example run:

chrome 108               ✓ components/test/wpt-app.html » <wpt-app> » WPTApp.prototype.* » computePathIsRootDir  » root dir
chrome 108               Tests failed: Error thrown outside of test function: uncaptured fetch: http://localhost:8081/api/versions?product=chrome
                             window.fetch at wpt-app.html:21
        ProductBuilder.versionsURLUpdated at /components/wptdashboard/components/product-builder.js:235
         Object.runObserverEffect [as fn] at /components/@polymer/polymer/lib/mixins/property-effects.js:245
                               runEffects at /components/@polymer/polymer/lib/mixins/property-effects.js:146
        ProductBuilder._propertiesChanged at /components/@polymer/polymer/lib/mixins/property-effects.js:2160
          ProductBuilder._flushProperties at /components/@polymer/polymer/lib/mixins/properties-changed.js:424
                     ProductBuilder.ready at /components/@polymer/polymer/lib/mixins/property-effects.js:2106
                     ProductBuilder.ready at /components/@polymer/polymer/lib/mixins/element-mixin.js:759
         ProductBuilder._enableProperties at /components/@polymer/polymer/lib/mixins/properties-changed.js:399
  TemplateInstance.__enableOrFlushClients at /components/@polymer/polymer/lib/mixins/property-effects.js:2038
chrome 108               CALL quit()
chrome 108               DELETE /session/:sessionID
chrome 108               RESPONSE quit()
chrome 108               BrowserRunner complete
Test run ended in failure: Error thrown outside of test function: uncaptured fetch: http://localhost:8081/api/versions?product=chrome
                             window.fetch at wpt-app.html:21
        ProductBuilder.versionsURLUpdated at /components/wptdashboard/components/product-builder.js:235
         Object.runObserverEffect [as fn] at /components/@polymer/polymer/lib/mixins/property-effects.js:245
                               runEffects at /components/@polymer/polymer/lib/mixins/property-effects.js:146
        ProductBuilder._propertiesChanged at /components/@polymer/polymer/lib/mixins/property-effects.js:2160
          ProductBuilder._flushProperties at /components/@polymer/polymer/lib/mixins/properties-changed.js:424
                     ProductBuilder.ready at /components/@polymer/polymer/lib/mixins/property-effects.js:2106
                     ProductBuilder.ready at /components/@polymer/polymer/lib/mixins/element-mixin.js:[759](https://github.com/web-platform-tests/wpt.fyi/actions/runs/3789867524/jobs/6444124312#step:3:760)
         ProductBuilder._enableProperties at /components/@polymer/polymer/lib/mixins/properties-changed.js:399
  TemplateInstance.__enableOrFlushClients at /components/@polymer/polymer/lib/mixins/property-effects.js:2038

Can compare it to when the tests work

It takes up unncessary time having to re-run the job constantly when the changed code does not affected that code or even that stack.

[jcscottiii]

The component test suite that is flaky is expecting no fetches to happen at all.

<wpt-app> imports test-runs-query-builder. That imports product-builder. Which leads to the stacktrace above. Whenever the url changes, it does a fetch.

https://github.com/web-platform-tests/wpt.fyi/blob/c96d81c013414e912bb21b25586630c31df02102/webapp/components/product-builder.js#L227-L229

[jcscottiii]

Because it happens intermittently, my hypothesis is:

• Sometimes the versionsURLUpdated observer is not triggered before the test finishes. In that case, it passes. If it does trigger, it fails.