chromaui / chromatic-e2e

Archive end-to-end tests to be replayed in Storybook and Chromatic
MIT License
13 stars 4 forks source link

Network idle watcher for Cypress #173

Closed skitterm closed 1 week ago

skitterm commented 1 week ago

Issue: #AP-3843

What Changed

Created a NetworkIdleWatcher class for Cypress that approximates what page.waitForLoadState('networkIdle') does in Playwright.

It's a very simplistic class since we're only using it after the Cypress test has run, to wait longer if there are any pending requests.

Limitations

This class isn't used anywhere yet. In a following PR I'll hook it up so the onRequest and onResponse callbacks are accepted as parameters into the ResourceArchiver() constructor, so that when the CDP detects requests or responses, there's a callback to the NetworkIdleWatcher. Then, when the test is done, we'll call await networkIdleWatcher.idle(), like we do for Playwright.

How to test

Run the unit tests! As in, have CI run them for you 😜

codacy-production[bot] commented 1 week ago

Coverage summary from Codacy

See diff coverage on Codacy

Coverage variation Diff coverage
:white_check_mark: +0.31% (target: -1.00%) :white_check_mark: 100.00% (target: 80.00%)
Coverage variation details | | Coverable lines | Covered lines | Coverage | | ------------- | ------------- | ------------- | ------------- | | Common ancestor commit (64f1936a5ec7427cb9bb4344b4c41fef18e3754b) | 260 | 246 | 94.62% | | | Head commit (604ef926dd5c3271149f5f56954c95be90b2e9c7) | 276 (+16) | 262 (+16) | 94.93% (**+0.31%**) | **Coverage variation** is the difference between the coverage for the head and common ancestor commits of the pull request branch: ` - `
Diff coverage details | | Coverable lines | Covered lines | Diff coverage | | ------------- | ------------- | ------------- | ------------- | | Pull request (#173) | 16 | 16 | **100.00%** | **Diff coverage** is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: `/ * 100%`

See your quality gate settings    Change summary preferences


:rocket: Don’t miss a bit, follow what’s new on Codacy.

Codacy stopped sending the deprecated coverage status on June 5th, 2024. Learn more

skitterm commented 1 week ago

What do you mean by that? Snapshots happen during a test run, so how would that help ensure that everything is loaded?

@tevanoff good question. I meant that we're only checking for idleness after the test is run, not during. For the time being, that enabled me to avoid doing a more elaborate idle-watcher (like declaring things idle if more than 200ms has elapsed between requests) and instead stuck with just waiting until all requests have responses (unless the timeout is hit).

tevanoff commented 1 week ago

@tevanoff good question. I meant that we're only checking for idleness after the test is run, not during. For the time being, that enabled me to avoid doing a more elaborate idle-watcher (like declaring things idle if more than 200ms has elapsed between requests) and instead stuck with just waiting until all requests have responses (unless the timeout is hit).

@skitterm Does cypress tear down the browser after the test is run? I guess I'm wondering if any requests that are still in flight may just be canceled after the test finishes, leaving us in the same state of not having the assets archived. Is that a possibility?

skitterm commented 1 week ago

@skitterm Does cypress tear down the browser after the test is run? I guess I'm wondering if any requests that are still in flight may just be canceled after the test finishes, leaving us in the same state of not having the assets archived. Is that a possibility?

@tevanoff that's a fair point. I'm operating on the assumption that the browser is still around when I check idleness (which starts from here), but I'll confirm if that is the case.

skitterm commented 1 week ago

@tevanoff I checked (by manually delaying an image request in our test server, and manually delaying in our Cypress plugin to stall for the same amount of time), and the image is archived! So we should be able to archive all resources without Cypress cutting off requests.