Open IanVS opened 11 months ago
Hey @IanVS thanks for opening this issue! I'll elaborate things a bit so others might have enough context when looking at this issue in the future.
The test-runner works by accessing the "iframe" URL of your Storybook, which has just a barebones, isolated version of Storybook with no stories, for instance like here. After that, it accesses Storybook specific APIs to select the story it needs to test. If it can't find Storybook APIs (in this case, the Storybook channel), it will error with that message you got.
First, it's important to know whether you're running the test-runner against a local Storybook or a deployed Storybook. It might also help to know whether this happens only in CI or not.
Second, this can mean a few things:
If it's none of the above, I really would like to know how to reproduce this so we can fix it!
Thanks Yann! I've only seen it in CI, but honestly I don't run the test-runner locally because I get very inconsistent failures and timeouts, so I nearly always push to CI and let it run there. Some other notes:
These are the scripts I use to run the tests in CI:
"test:storybook:ci": "run-p --race test:storybook preview-storybook",
"test:storybook": "wait-on http://127.0.0.1:6006 && test-storybook --url http://127.0.0.1:6006 -- --shard=$SHARD",
"preview-storybook": "http-server storybook-static --port 6006 --silent",
I'll keep my eyes open for any further cases, and see if I can find any similarities.
I'm also getting this if I run test-runner with--browsers webkit
it's mostly just happening on webkit
.
I also get:
FAIL browser: webkit stories/components/Tabs.stories.tsx
[TEST] ● Test suite failed to run
[TEST]
[TEST] page.goto: Timeout 30000ms exceeded.
[TEST] =========================== logs ===========================
[TEST] navigating to "http://127.0.0.1:6006/iframe.html", waiting until "load"
[TEST] ============================================================
[TEST]
[TEST] at defaultPrepare (node_modules/@storybook/test-runner/dist/index.js:13241:14)
[TEST] at setupPage (node_modules/@storybook/test-runner/dist/index.js:13273:11)
[TEST] at CustomEnvironment.setup (node_modules/@storybook/test-runner/playwright/custom-environment.js:8:5)
[TEST]
[TEST] FAIL browser: webkit stories/components/Skeleton.stories.tsx (121.925 s)
[TEST] ● Components/Skeleton › WithText › smoke-test
[TEST]
[TEST] thrown: "Exceeded timeout of 30000 ms for a test.
[TEST] Add a timeout value to this test to increase the timeout, if this is a long-running test. See https://jestjs.io/docs/api#testname-fn-timeout."
[TEST]
[TEST] > 1 | import React from 'react';
[TEST] |
I tried with both a dev instance of storybook + a built one. this error was with the built one.
chromium
works fine. I also increased the timeout but didn't help.
These stories have nothing special, just smoke-tests.
I'm going to try and set up a repo where I can replicate it but might take some time.
We use vite for our build, not sure if that helps.
I also face similar issue as @pedrosousa13:
[TEST] FAIL packages/react-ui/src/components/Input/Input.stories.tsx
● Test suite failed to run
[TEST]
page.goto: Timeout 30000ms exceeded.
[TEST] Call log:
- navigating to "http://localhost:7006/iframe.html", waiting until "load"
[TEST]
at defaultPrepare (node_modules/.pnpm/@storybook+test-runner@0.19.1_@swc+helpers@0.5.11_@types+node@17.0.45_babel-plugin-macros@3.1_27kwgapvhdmhdh2inwdzlb3z3e/node_modules/@storybook/test-runner/dist/index.js:13241:14)
at setupPage (node_modules/.pnpm/@storybook+test-runner@0.19.1_@swc+helpers@0.5.11_@types+node@17.0.45_babel-plugin-macros@3.1_27kwgapvhdmhdh2inwdzlb3z3e/node_modules/@storybook/test-runner/dist/index.js:13273:11)
at CustomEnvironment.setup (node_modules/.pnpm/@storybook+test-runner@0.19.1_@swc+helpers@0.5.11_@types+node@17.0.45_babel-plugin-macros@3.1_27kwgapvhdmhdh2inwdzlb3z3e/node_modules/@storybook/test-runner/playwright/custom-environment.js:8:5)
I am also using vite
.
This occurs when I am doing snapshot testing using jest-image-snapshot
.
I suspect it is related to postVisit
.
Does test-runner
wait for the previous story's postVisit
before starting a new one?
I suspect page
has a promise queue and it got piled up.
In my case, I am running test-storybook
with about 1k stories.
concurrently -k -s first -n \"SB,TEST\" -c \"magenta,blue\" \"storybook dev -p 7006 --ci\" \"wait-on http://localhost:7006 && test-storybook --url http://localhost:7006 -u
The snapshot test occurs in the postVisit
, and all promises are awaited:
// simplified for illustrative purpose
(
...,
async postVisit(page, context) {
await ctx.waitForPageReady(page)
const image = await page.screenshot()
expect(image).toMatchImageSnapshot(...)
await switchTheme(page, 'dark')
const image = await page.screenshot()
expect(image).toMatchImageSnapshot(...)
}
could be related to #444
Describe the bug I'm not sure what can be done to fix this, but I've gotten an error in a single test, which was resolved upon a re-run of the job:
To Reproduce No idea, it's not something I've seen often, and appears to be intermittent. The story is nothing special: no args, no play function, just static content.
Expected behavior I wouldn't expect flakey behavior from the test runner.
Screenshots If applicable, add screenshots to help explain your problem.
System
Additional context This isn't probably immediately actionable, but I wanted to open an issue to see if others are hitting this as well, in which case we might want to look into why it could happen.