Web test logs display errors but are not failing

DanielRyanSmith commented 1 year ago

When running npm run webtest on main branch, these logs appear:

 🚧 Browser logs:
      An error was thrown in a Promise outside a test. Did you forget to await a function or assertion?
 🚧 Browser logs on Chromium:
      TypeError: Cannot read properties of null (reading 'showMessage')
        at showToastMessage (client-src/elements/utils.js:19:11)
        at client-src/elements/chromedash-activity-page.js:79:7
 🚧 Browser logs on Firefox:
      TypeError: toastEl is null
        at showToastMessage (client-src/elements/utils.js:19:3)
        at fetchComments/< (client-src/elements/chromedash-activity-page.js:79:23)
        at promise callback*fetchComments (client-src/elements/chromedash-activity-page.js:78:8)
        at connectedCallback (client-src/elements/chromedash-activity-page.js:67:10)
        at S (node_modules/lit-html/src/lit-html.ts:1402:52)
        at T (node_modules/lit-html/src/lit-html.ts:1445:35)
        at $ (node_modules/lit-html/src/lit-html.ts:1551:11)
        at _$AI (node_modules/lit-html/src/lit-html.ts:1375:11)
        at B (node_modules/lit-html/src/lit-html.ts:2172:7)
        at litFixtureSync (node_modules/@open-wc/testing-helpers/src/litFixture.js:32:9)
        at litFixture (node_modules/@open-wc/testing-helpers/src/litFixture.js:63:28)
        at fixture (node_modules/@open-wc/testing-helpers/src/fixture-no-side-effect.js:62:12)
        at client-src/elements/chromedash-activity-page_test.js:21:29

The tests do not actually fail, so whatever change is causing this was merged without being flagged. The errors need to be investigated to determine the root cause and fixed.

DanielRyanSmith commented 8 months ago

This problem continues to worsen. Running npm run webtest today on the main branch looks like this:


 🚧 Browser logs:
      An error was thrown in a Promise outside a test. Did you forget to await a function or assertion?
 🚧 Browser logs on Chromium:
      TypeError: Cannot read properties of null (reading 'showMessage')
        at showToastMessage (client-src/elements/utils.js:31:11)
        at client-src/elements/chromedash-activity-page.js:79:7
 🚧 Browser logs on Firefox:
      TypeError: toastEl is null
        at showToastMessage (client-src/elements/utils.js:31:3)
        at fetchComments/< (client-src/elements/chromedash-activity-page.js:79:23)
        at promise callback*fetchComments (client-src/elements/chromedash-activity-page.js:78:8)
        at connectedCallback (client-src/elements/chromedash-activity-page.js:67:10)
        at k (node_modules/lit-html/src/lit-html.ts:1411:52)
        at $ (node_modules/lit-html/src/lit-html.ts:1454:35)
        at g (node_modules/lit-html/src/lit-html.ts:1563:11)
        at _$AI (node_modules/lit-html/src/lit-html.ts:1384:11)
        at D (node_modules/lit-html/src/lit-html.ts:2183:7)
        at litFixtureSync (node_modules/@open-wc/testing-helpers/src/litFixture.js:32:9)
        at litFixture (node_modules/@open-wc/testing-helpers/src/litFixture.js:63:28)
        at fixture (node_modules/@open-wc/testing-helpers/src/fixture-no-side-effect.js:62:12)
        at client-src/elements/chromedash-activity-page_test.js:21:29

client-src/elements/chromedash-feature-page_test.js:

 🚧 Browser logs:
      An error was thrown in a Promise outside a test. Did you forget to await a function or assertion?
      An error was thrown in a Promise outside a test. Did you forget to await a function or assertion?
      An error was thrown in a Promise outside a test. Did you forget to await a function or assertion?
      An error was thrown in a Promise outside a test. Did you forget to await a function or assertion?
      An error was thrown in a Promise outside a test. Did you forget to await a function or assertion?
      An error was thrown in a Promise outside a test. Did you forget to await a function or assertion?
 🚧 Browser logs on Chromium:
      ChromeStatusHttpError: Got error response from server /feature_links?feature_id=0&update_stale_links=true: 404
        at ChromeStatusClient.doFetch (client-src/js-src/cs-client.js:91:13)
      ChromeStatusHttpError: Got error response from server /feature_links?feature_id=0&update_stale_links=true: 404
        at ChromeStatusClient.doFetch (client-src/js-src/cs-client.js:91:13)
      ChromeStatusHttpError: Got error response from server /feature_links?feature_id=123456&update_stale_links=true: 404
        at ChromeStatusClient.doFetch (client-src/js-src/cs-client.js:91:13)
      ChromeStatusHttpError: Got error response from server /feature_links?feature_id=123456&update_stale_links=true: 404
        at ChromeStatusClient.doFetch (client-src/js-src/cs-client.js:91:13)
      ChromeStatusHttpError: Got error response from server /feature_links?feature_id=123456&update_stale_links=true: 404
        at ChromeStatusClient.doFetch (client-src/js-src/cs-client.js:91:13)
      ChromeStatusHttpError: Got error response from server /feature_links?feature_id=123456&update_stale_links=true: 404
        at ChromeStatusClient.doFetch (client-src/js-src/cs-client.js:91:13)
 🚧 Browser logs on Firefox:
      ChromeStatusHttpError: Got error response from server /feature_links?feature_id=0&update_stale_links=true: 404
        at ChromeStatusHttpError (client-src/js-src/cs-client.js:25:5)
        at doFetch (client-src/js-src/cs-client.js:91:13)
        at async*doGet (client-src/js-src/cs-client.js:108:17)
        at getFeatureLinks (client-src/js-src/cs-client.js:313:17)
        at fetchData (client-src/elements/chromedash-feature-page.js:176:21)
        at connectedCallback (client-src/elements/chromedash-feature-page.js:132:10)
        at k (node_modules/lit-html/src/lit-html.ts:1411:52)
        at $ (node_modules/lit-html/src/lit-html.ts:1454:35)
        at g (node_modules/lit-html/src/lit-html.ts:1563:11)
        at _$AI (node_modules/lit-html/src/lit-html.ts:1384:11)
        at D (node_modules/lit-html/src/lit-html.ts:2183:7)
        at litFixtureSync (node_modules/@open-wc/testing-helpers/src/litFixture.js:32:9)
        at litFixture (node_modules/@open-wc/testing-helpers/src/litFixture.js:63:28)
        at fixture (node_modules/@open-wc/testing-helpers/src/fixture-no-side-effect.js:62:12)
        at client-src/elements/chromedash-feature-page_test.js:146:29
      ChromeStatusHttpError: Got error response from server /feature_links?feature_id=123456&update_stale_links=true: 404
        at ChromeStatusHttpError (client-src/js-src/cs-client.js:25:5)
        at doFetch (client-src/js-src/cs-client.js:91:13)
        at async*doGet (client-src/js-src/cs-client.js:108:17)
        at getFeatureLinks (client-src/js-src/cs-client.js:313:17)
        at fetchData (client-src/elements/chromedash-feature-page.js:176:21)
        at connectedCallback (client-src/elements/chromedash-feature-page.js:132:10)
        at k (node_modules/lit-html/src/lit-html.ts:1411:52)
        at $ (node_modules/lit-html/src/lit-html.ts:1454:35)
        at g (node_modules/lit-html/src/lit-html.ts:1563:11)
        at _$AI (node_modules/lit-html/src/lit-html.ts:1384:11)
        at D (node_modules/lit-html/src/lit-html.ts:2183:7)
        at litFixtureSync (node_modules/@open-wc/testing-helpers/src/litFixture.js:32:9)
        at litFixture (node_modules/@open-wc/testing-helpers/src/litFixture.js:63:28)
        at fixture (node_modules/@open-wc/testing-helpers/src/fixture-no-side-effect.js:62:12)
        at client-src/elements/chromedash-feature-page_test.js:178:29

 🚧 404 network requests:
    - api/v0/feature_links?feature_id=0&update_stale_links=true
    - api/v0/feature_links?feature_id=123456&update_stale_links=true

dlaliberte commented 8 months ago

I'm seeing these errors too, since I am running the webtest before adding a new test for the semantic checks. I think I see how we can clean up most of these. Not sure yet about the 'update_stale_links' URLs.

GoogleChrome / chromium-dashboard

Web test logs display errors but are not failing #2977