Chromatic playwright initial build/deploy hangs

[tteltrab]

Describe the bug

Running into two issues on first setup with playwright (not sure if related, happy to open a second issue if unrelated).

1. On initial setup of a playwright app for chromatic, the build seems to hang indefinitely to timeout.

I've tracked this initial problem down to what I think is an issue with the interactive storybook ask for anonymous crash reports. If I re-run the storybook build command from the chromatic.log locally ( node C:\Users\nbart\dev\chromatic-test\nodemodules.pnpm\@chromatic-com+playwright@0.6.7@babel+preset-env@7.24.5_@babel+core@7.24.5__@playwright+test_5olcbfv4pgsjbfkqjknq6pmjmu\node_modules\@chromatic-com\playwright\dist\bin\build-archive-storybook.js --output-dir C:\Users\nbart\AppData\Local\Temp\chromatic--70996-63Igu9KkH6w3 )

I get prompted the first time and it hangs:

I then tell it I do/don't want to send reports, and on the second time it fails almost immediately (no hanging at least).

Re-running chromatic --playwright no longer hangs, but:

2. Initial build/deploy fails with error

Linked logs and example repo at the bottom. Screenshots for reference.

Steps to reproduce

git init
pnpm init
pnpm create playwright

# commit/push to a github repo

pnpm install --save-dev chromatic @chromatic-com/playwright

# update import in tests/example.spec.ts to import from "@chromatic-com/playwright". commit and push

pnpm playwright test # this should work fine locally
pnpm chromatic --playwright --project-token=chpt_ed077c727... # this hangs

Additional context I could be missing something obvious in the setup, it's my first time setting up playwright with chromatic. I followed the steps in the guide, and created a basic example repo to see if it was specific to the one project, but ran into it in the basic example too.

It could be something specific to my machine or setup - running local windows machine, not on CI. Using pnpm but tried with npm and ran into a similar issue.

• Basic repo for reproduction

I've changed the token since. Also FWIW, I got storybook with chromatic in a separate project working without issue, so it seems to be playwright-specific.

Logs for reference: chromatic-fail-after-timeout-fix.log chromatic-timeout.log chromatic-diagnostics.json build-storybook.log

[jackie-linz]

We are see the same problem when upgrading from

• @chromatic-com/playwright@0.5.4
• chromatic@10.8.0

to

• @chromatic-com/playwright@0.6.7 (also tried 0.6.0, same issue)
• chromatic@11.3.5

[thafryer]

Hi Everyone! Thanks for reporting! We will update you when we have more info!

[artmsilva]

We're also experiencing a similar issue. Nothing is saved to ./test-results/chromatic-archives.

[terrymun]

We are also experiencing the same, currently using @chromatic-com/playwright@0.6.7. We have a monorepo setup so the E2E test results are stored in different location, but we have ensured that CHROMATIC_ARCHIVE_LOCATION env variable is set.

Running npx chromatic --playwright -t=<token> times out at the "Building your storybook" step. I've reached out in chat but was asked to use the --zip flag, but that shouldn't change anything since we have a one-liner E2E in our current setup as a POC, so definitely not a size issue.

I can confirm there is indeed files being written to the chromatic-archives folder:

I also tried using the last known working combination of @chromatic-com/playwright@0.5.4 and chromatic@10.8.0 that @jackie-linz reported to work for him, but to no avail.

[thafryer]

Hi Folks! One of the downstream dependencies for our process of generating the archives had a bug that has since been resolved. We've released some changes that should resolve this issue for you all! Please update to @chromatic-com/playwright@0.6.11 and chromatic@11.4.1!

Let me know if you are still running into issues!

[terrymun]

Thanks for the update @thafryer! I can confirm that after upgrading to the latest versions of @chromatic-com/playwright and chromatic, it is possible to publish playwright-generated snapshots via the CLI again 👏

[markmetcalfe]

@thafryer I've upgraded my project to use @chromatic-com/playwright@0.6.11 and chromatic@11.5.3 but I'm still finding that the test-results/chromatic-archives directory is not being generated when running playwright tests.

I get the following error output when running npx chromatic --playwright (after running npx playwright test):

[terrymun]

@markmetcalfe Before you run the chromatic command you need to have already run playwright beforehand. Are you sure you've done that? It's not clear in the screenshot that a playwright test has already been run.

If anything, what you're experience likely has nothing to do with the bug being reported here, as it deals with an issue with the chromatic command not reading the files correctly (instead of not finding the output).

[markmetcalfe]

@terrymun I've run the npx playwright test command beforehand. As you say, my issue is likely unrelated. I've opened #155