chromaui / chromatic-e2e

Archive end-to-end tests to be replayed in Storybook and Chromatic
MIT License
21 stars 4 forks source link

:goal_net: Make Watcher not throw an exception if network idle times out #25

Closed jwir3 closed 11 months ago

jwir3 commented 11 months ago

Fixes AP-3761.

Issue: AP-3761

What Changed

Previous to this, Watcher would throw an exception if the global network idle timer timed out. This changes it to not throw an error, and instead log the error and move on. It also adjusts the default timeout to be configurable, defaulting to 10s.

How to test

Follow the reproduction steps in AP-3761.

Change Type

📦 Published PR as canary version: 0.0.29--canary.25.4e99d70.0
:sparkles: Test out this PR locally via: ```bash npm install @chromaui/test-archiver@0.0.29--canary.25.4e99d70.0 # or yarn add @chromaui/test-archiver@0.0.29--canary.25.4e99d70.0 ```
linear[bot] commented 11 months ago
AP-3761 CDP failures are causing tests to fail

**What** A [customer](https://chromaticqa.slack.com/archives/C05L6F1TYEA/p1697555064653969) sees tests failing when using `chromaui/test-archiver` that aren't failing when only using `playwright-test`. test-archiver is timing out after a specified interval (`2000 ms` currently), and some of the resources (`.woff` font files) are taking longer than that to load, so we're throwing an [error](https://github.com/chromaui/test-archiver/blob/main/src/resource-archive/index.ts#L66), which fails the spec. **Reproducing the error** I can reproduce by putting the following spec into a typical Playwright project, then running `npx playwright test`: ``` test('has title', async ({ page }, testInfo) => { await page.goto('https://www.getty.edu/art/collection/search?images=true&q=sculpture'); // Expect a title "to contain" a substring. await expect(page).toHaveTitle(/Search for keyword: 'sculpture', with Images \(Getty Museum\)/); }); ``` **Possible solutions** I don't experience the problem if I increase the [network timeout](https://github.com/chromaui/test-archiver/blob/main/src/resource-archive/index.ts#L5) (anything above `5000 ms`). Not sure if there's a reason it is currently at `2000 ms`, though (perhaps that makes the tests finish more quickly if all is well).

thafryer commented 11 months ago

:rocket: PR was released in v0.0.29 :rocket: