storybookjs / test-runner

🚕 Turn stories into executable tests
https://storybook.js.org/docs/writing-tests/interaction-testing
MIT License
220 stars 66 forks source link

[Bug]: --includeTags missing some tests #484

Closed kraftman closed 3 weeks ago

kraftman commented 1 month ago

Describe the bug

I'm using --includeTags to filter components tagged with 'component'. While the tests are running, it says '24 passed', but when it completes it changes to '13 passed', and ignores some of the component stories. I've tried checking the stories to see if there are any obvious differences but I cant see any. I've included examples below.

To Reproduce

run :test-storybook --includeTags=\"component\" --coverage --coverageDirectory sb-coverage

this story runs:

import React from 'react';
import { userEvent, within, expect, fn } from '@storybook/test';
import CancelButton from './CancelButton';

export default {
  title: 'Components/CancelButton',
  tags: ['component'],
  component: CancelButton,
};

export const Default = {};

export const Test = {
  args: {
    onClick: fn(),
    onClickData: { test: 'test' },
  },
  play: async ({ args, canvasElement }) => {
    const canvas = within(canvasElement);
    await userEvent.click(canvas.getByRole('button'));
    await expect(args.onClick).toHaveBeenCalledWith({ test: 'test' });
  },
};

this story is ignored:

import React from 'react';
import { within, expect, fn, userEvent } from '@storybook/test';
import Checkbox from './Checkbox';

export default {
  title: 'Components/Checkbox',
  component: Checkbox,
  tags: ['component'],
  args: {
    onChange: fn(),
    defaultValue: false,
    label: 'Send me news',
    value: false,
    formState: {
      checkbox: {
        value: {
          checkbox: false,
        },
      },
    },
    componentName: 'checkbox',
  },
};

export const Default = {
  play: async ({ args, canvasElement }) => {
    const canvas = within(canvasElement);
    const checkbox = canvas.getByRole('checkbox');
    expect(checkbox).toBeInTheDocument();
    userEvent.click(checkbox);
    // await waitFor(() => expect(checkbox).toBeChecked());
    // expect(args.onChange).toHaveBeenCalledWith({ value: { checkbox: true } });
  },
};

System

Storybook Environment Info:

  System:
    OS: Linux 5.15 Ubuntu 22.04.2 LTS 22.04.2 LTS (Jammy Jellyfish)
    CPU: (24) x64 AMD Ryzen 9 7900X 12-Core Processor
    Shell: 5.8.1 - /usr/bin/zsh
  Binaries:
    Node: 20.13.0 - ~/.nvm/versions/node/v20.13.0/bin/node
    Yarn: 1.22.22 - ~/mason-web/node_modules/.bin/yarn <----- active
    npm: 10.5.2 - ~/.nvm/versions/node/v20.13.0/bin/npm
  npmPackages:
    @storybook/addon-actions: ^8.1.9 => 8.1.9
    @storybook/addon-controls: ^8.1.9 => 8.1.9
    @storybook/addon-coverage: ^1.0.4 => 1.0.4
    @storybook/addon-designs: ^8.0.2 => 8.0.2
    @storybook/addon-essentials: ^8.1.9 => 8.1.9
    @storybook/addon-interactions: ^8.1.9 => 8.1.9
    @storybook/addon-links: ^8.1.9 => 8.1.9
    @storybook/addon-mdx-gfm: ^8.1.9 => 8.1.9
    @storybook/addon-storysource: ^8.1.9 => 8.1.9
    @storybook/addon-viewport: ^8.1.9 => 8.1.9
    @storybook/addon-webpack5-compiler-babel: ^3.0.3 => 3.0.3
    @storybook/manager-api: ^8.1.9 => 8.1.9
    @storybook/react: ^8.1.9 => 8.1.9
    @storybook/react-webpack5: ^8.1.9 => 8.1.9
    @storybook/source-loader: ^8.1.9 => 8.1.9
    @storybook/test: ^8.1.9 => 8.1.9
    @storybook/test-runner: ^0.18.2 => 0.18.2
    msw-storybook-addon: ^2.0.2 => 2.0.2
    storybook: ^8.1.9 => 8.1.9

Additional context

No response

kraftman commented 3 weeks ago

So I dug into this a bit more, and it seems to be an issue about only having one named export? or one named export with tests?

if I change 'export const Default' to 'export const Test' it still isnt picked up, however if i then add an empty 'export const Default = {}' above it then the test is included

for reference this is the updated version of the above that works

import React from 'react';
import { within, expect, fn, userEvent, waitFor } from '@storybook/test';
import Checkbox from './Checkbox';

export default {
  title: 'Components/Checkbox',
  tags: ['component'],
  component: Checkbox,
  args: {
    onChange: fn(),
    defaultValue: false,
    label: 'Send me news',
    value: false,
    formState: {
      checkbox: {
        value: {
          checkbox: false,
        },
      },
    },
    componentName: 'checkbox',
  },
};

export const Default = {};

export const ValidateClick = {
  play: async ({ args, canvasElement }) => {
    const canvas = within(canvasElement);
    const checkbox = canvas.getByRole('checkbox');
    expect(checkbox).toBeInTheDocument();
    userEvent.click(checkbox);
    await waitFor(() => expect(checkbox).toBeChecked());
    expect(args.onChange).toHaveBeenCalledWith({ value: { checkbox: true } });
  },
};
yannbf commented 3 weeks ago

Hey @kraftman thanks a lot for catching this bug. There is an inconsistency between transformation modes (non index-json and index-json), I'll work on fixing this ASAP!