[RFC]: Storybook Storyshots are working, can we get core support?

toxsick commented 2 years ago

Summary

There is #2064 for extended config features around storybook and it explicitly motions stroryshots:

Storyshots & Jest Those two really make sense since RW ships with Jest since... can't recall x). Storyshots would let us track changes in the markup, it's not worth Chromatic which handles visual testing but this one is cloud based and pricey ( well, there's a generous free plan, but it still is cloud based ). Anyway, we have today a codebase that changes a lot because the project is young, this would help us stabilize it. The Jest addon on the other hand should provide reports of code coverage, I'm not sure exactly what else we can gather with it but I think it's worth digging.

I think this is possible right now.

Motivation

To get storyshots support in redwood core.

Detailed proposal

Here is my current setup to get storyshots working with redwood:

// web/package.json

  "devDependencies": {
     ...
    "@storybook/addon-storyshots": "6.5.5",
    "identity-obj-proxy": "3.0.0",
     ...
  }
}

// web/jest.config.js

// More info at https://redwoodjs.com/docs/project-configuration-dev-test-build

const config = {
  rootDir: '../',
  preset: '@redwoodjs/testing/config/jest/web',
  verbose: true,
  moduleNameMapper: {
    // mock redwood magic webpack aliases
    // see: https://github.com/redwoodjs/redwood/blob/main/packages/testing/config/storybook/preview.js
    '~__REDWOOD__USER_WEB_DEFAULT_CSS': 'identity-obj-proxy',
    '~__REDWOOD__USER_STORYBOOK_PREVIEW_CONFIG':
      '<rootDir>/web/config/storybook.preview.js',
  },
  setupFilesAfterEnv: [`<rootDir>/web/test/mocks.ts`],
};

module.exports = config;

// web/src/Storyshots.spec.ts

import path from 'path';
import initStoryshots, {
  multiSnapshotWithOptions,
} from '@storybook/addon-storyshots';
import { render } from '@testing-library/react';

const reactTestingLibrarySerializer = {
  print: (val, serialize) => serialize(val.container.firstChild),
  test: (val) => val && Object.prototype.hasOwnProperty.call(val, 'container'),
};

initStoryshots({
  framework: 'react',
  configPath: path.resolve(
    __dirname,
    '../../node_modules/@redwoodjs/testing/config/storybook',
  ),
  test: (story) => {
    // FIXME Workaround for https://github.com/storybookjs/storybook/issues/16692
    const fileName = path.resolve(
      __dirname,
      '../../',
      story.context.fileName,
    );
    return multiSnapshotWithOptions({
      renderer: render,
      snapshotSerializers: [reactTestingLibrarySerializer],
    })({
      ...story,
      context: { ...story.context, fileName },
    });
  },
});

// web/test/mocks.ts

// fix for 'ResizeObserver is not defined' in storyshots
// see: https://jestjs.io/docs/manual-mocks#mocking-methods-which-are-not-implemented-in-jsdom
// see: https://github.com/que-etc/resize-observer-polyfill/issues/50#issuecomment-998746784
Object.defineProperty(window, 'ResizeObserver', {
  writable: true,
  value: jest.fn().mockImplementation(() => ({
    observe: jest.fn(() => 'Mocking works'),
    unobserve: jest.fn(),
    disconnect: jest.fn(),
  })),
});

Are you interested in working on this?

[ ] I'm interested in working on this

noire-munich commented 2 years ago

Thanks a lot for this @toxsick, this was indeed mentioned in an old draft. Glad to see someone else has needed this enough to have some code!

Looping in @virtuoushub, can you have a look at this?

virtuoushub commented 2 years ago

🌊 @toxsick - I have used the information you provided (thank you so much!) and implemented this over at https://github.com/redwoodjs/example-storybook/pull/115

One thing to note is that it seems to be a bit finicky if there is a test failure. If ay test fails, it looks like jest does not run other tests. Need to look more into what is going on, but initial testing seems very promising.

Will report back, once I know more.

jtoar commented 2 years ago

One thing to note is that it seems to be a bit finicky if there is a test failure. If ay test fails, it looks like jest does not run other tests

@virtuoushub I have no context, but is it possible there's a failFast setting or something like that that's enabled?

virtuoushub commented 2 years ago

One thing to note is that it seems to be a bit finicky if there is a test failure. If ay test fails, it looks like jest does not run other tests

@virtuoushub I have no context, but is it possible there's a failFast setting or something like that that's enabled?

hard to say, there is so much webpack/storybook/react/jest abstraction layers at play that I need to spend a bit more time to know more.

virtuoushub commented 2 years ago

fwiw, got this working over at: https://github.com/redwoodjs/example-storybook

relevant code:

my main take away is that it is not ready for core. Making this more of a userland decision, with documentation to help those who want to get it working is the path I recommend.

jtoar commented 1 year ago

As @virtuoushub said, we won't be integrating this yet. That doesn't preclude us from doing so in the future though—most likely after storybook v7. But it's possible to set up right now using the example he's pointed to.

redwoodjs / redwood