Smart story names (path, filename, describes, test title)

skitterm commented 5 months ago

Issue: #AP-4015

What Changed

For the story names, we now use more than just the test title. Previously, we've been only using the Playwright/Cypress test title (a Playwright test with test('some-title') would have the story name some-title), which resulted in no hierarchy and possible overwriting since tests in different files with the same title would collide into one story name.

Now, we use the test title, then any containing describe()s as folders, then the filename as a folder (truncating the .spec.* or .cy.* stuff), then any directories between the filename and the tests directory as folders.

I'm trying to keep as similar to possible as how CSF 3 does auto-titling, so I'm not replacing _ or - with spaces, or capitalizing things, or doing any modifications besides stripping off the extension (.spec.ts|.cy.ts) of the file name. I'm not doing the redundancy-removing that CSF does, however (reducing MyComponent/MyComponent.stories.ts to MyComponent), since I bet it's unlikely that Playwright or Cypress users would folder their individual test files in folders of the same name. But I could be assuming too much there.

Example

Given the following Playwright test:

// tests/ is root tests directory
// this file path is tests/some-dir/file.spec.ts

test.describe('a containing describe', () => {
  test('test title', () => {
    // actual test stuff
  })
})

the resulting story name would be some-dir/file/a containing describe/test title.

Note

This is a breaking change for folks, as the first build they run after installing this version of the package will result in all-new snapshots (since all the story names will have changed).

It also means we don't have to manually add / to our test titles to get things foldered nicely in Storybook!

How to test

Check out the chromatic-e2e repo
In the Playwright and tests, add some test.describe() blocks around some test()s (describe()s around it()s for Cypress)
In a test title, add a period to the beginning of the title (like test('.someMethod() should behave'))
Run yarn build && yarn test:playwright && yarn test:cypress
Run yarn archive-storybook:playwright and yarn archive-storybook:cypress
Verify that the story names include the describe titles as folder names
Verify that the tests' filenames are folders
Verify that the test title with the period did not get truncated
Create a folder at the tests level
Move a tests file into this folder
Build, test, and archive-storybook again
Verify that this folder is included as a folder in Storybook for this test
[x] Author QA
[ ] Reviewer QA

skitterm commented 5 months ago

@vanessayuenn if you can have someone from the Storybook team look over how we create the titles (code and requirements -- see the "What Changed" section in the PR description), that'd be great. I want to make sure we doing this as similar to how CSF 3 auto-titles as possible.

skitterm commented 5 months ago

Storybook 8 actually does this a bit differently, only stripping off .stories.ts. So in SB 7, foo.bar.stories.ts would become foo, but in SB8 it's foo.bar. See https://github.com/storybookjs/storybook/blob/next/MIGRATION.md#autotitle-breaking-fixes

@JReinhold thanks so much for this feedback! I realized I can just strip off any .spec.ts|js (Playwright) and .cy.ts|js (Cypress) and thus accomplish the same thing, without truncating directories or test titles.

There's a slim possibility I'd miss some of the test files if someone can have a custom file extension for test files (not sure the test frameworks would allow that, though), but it's definitely better to err on the side of not truncating enough, since we can always add more rules later.

chromaui / chromatic-e2e