search

storybookjs / test-runner

🚕 Turn stories into executable tests
https://storybook.js.org/docs/writing-tests/interaction-testing
MIT License
231 stars 72 forks source link

"storybook test --coverage" is not printing coverage to console #454

Closed jnahumphreys closed 5 months ago

jnahumphreys commented 1 year ago

Describe the bug

On running npm test-storybook --coverage the coverage report is failing to print to the console (tests run fine btw)

Steps to reproduce the behavior

  1. Create a component built using TypeScript
  2. Write stories for said component
  3. start storybook npm run storybook
  4. run tests with coverage flag for storybook npm run storybook --coverage

Expected behavior

My tests should run and print the coverage report to my console once all tests are comple

Screenshots and/or logs

Ran all test suites.
Coverage file (134643 bytes) written to .nyc_output/coverage.json
nyc report

run coverage report for .nyc_output

Options:
  --cwd                   working directory used when resolving paths
                           [string] [default: "/Users/james/Development/Personal
                                           Projects/binaryjim-github-user-site"]
  --nycrc-path            specify an explicit path to find nyc configuration
  --temp-dir, -t, -t      directory to output raw coverage information to
                                             [string] [default: "./.nyc_output"]
  --exclude, -x, -x       a list of specific files and directories that should
                          be excluded from coverage, glob patterns are supported
                                                              [string] [default:
  ["coverage/**","packages/*/test{,s}/**","**/*.d.ts","test{,s}/**","test{,-*}.{
  js,cjs,mjs,ts,tsx,jsx}","**/*{.,-}test.{js,cjs,mjs,ts,tsx,jsx}","**/__tests__/
  **","**/{ava,babel,nyc}.config.{js,cjs,mjs}","**/jest.config.{js,cjs,mjs,ts}",
         "**/{karma,rollup,webpack}.config.js","**/.{eslint,mocha}rc.{js,cjs}"]]
  --exclude-node-modules  whether or not to exclude all node_module folders
                          (i.e. **/node_modules/**) by default
                                                       [boolean] [default: true]
  --include, -n, -n       a list of specific files that should be covered, glob
                          patterns are supported          [string] [default: []]
  --extension, -e, -e     a list of extensions that nyc should handle in
                          addition to .js
                   [string] [default: [".js",".cjs",".mjs",".ts",".tsx",".jsx"]]
  --exclude-after-remap   should exclude logic be performed after the source-map
                          remaps filenames?            [boolean] [default: true]
  --branches              what % of branches must be covered?
                                                           [number] [default: 0]
  --functions             what % of functions must be covered?
                                                           [number] [default: 0]
  --lines                 what % of lines must be covered?[number] [default: 90]
  --statements            what % of statements must be covered?
                                                           [number] [default: 0]
  --per-file              check thresholds per file   [boolean] [default: false]
  --check-coverage        check whether coverage is within thresholds provided
                                                      [boolean] [default: false]
  --reporter, -r, -r      coverage reporter(s) to use
                                                    [string] [default: ["text"]]
  --report-dir            directory to output coverage reports in
                                                  [string] [default: "coverage"]
  --show-process-tree     display the tree of spawned processes
                                                      [boolean] [default: false]
  --skip-empty            don't show empty files (no lines of code) in report
                                                      [boolean] [default: false]
  --skip-full             don't show files with 100% statement, branch, and
                          function coverage           [boolean] [default: false]
  -h, --help              Show help                                    [boolean]
  --version               Show version number                          [boolean]

Examples:
  nyc report --reporter=lcov  output an HTML lcov report to ./coverage

Too many non-option arguments: got 2, maximum of 0
/Users/james/Development/Personal Projects/binaryjim-github-user-site/node_modules/@storybook/test-runner/bin/test-storybook.js:26
  throw err;
  ^

Error: Command failed: npx nyc report --reporter=text -t /Users/james/Development/Personal Projects/binaryjim-github-user-site/coverage/storybook --report-dir /Users/james/Development/Personal Projects/binaryjim-github-user-site/coverage/storybook
    at checkExecSyncError (node:child_process:841:11)
    at execSync (node:child_process:912:15)
    at reportCoverage (/Users/james/Development/Personal Projects/binaryjim-github-user-site/node_modules/@storybook/test-runner/bin/test-storybook.js:73:3)
    at process.onProcessEnd (/Users/james/Development/Personal Projects/binaryjim-github-user-site/node_modules/@storybook/test-runner/bin/test-storybook.js:80:3)
    at process.emit (node:events:525:35)
    at process.el [as emit] (/Users/james/Development/Personal Projects/binaryjim-github-user-site/node_modules/playwright-core/lib/utilsBundleImpl.js:14:10212) {
  status: 1,
  signal: null,
  output: [ null, null, null ],
  pid: 75613,
  stdout: null,
  stderr: null
}

Environment

Additional context

My Typescript has been instrumented successfully, after running npm run storybook --coverage I can generate an lcov report by running npx nyc report --reporter=lcov -t coverage/storybook --report-dir coverage/storybook.

lucascurti commented 1 year ago

Installing nyc as a dev dependency fixed it for me.

Victor-Nyagudi commented 1 year ago

I'm experiencing the same issue on Windows.

Environment

I tried installing nyc as a dev dependency, but that didn't work either.

rbright commented 1 year ago

Just a heads-up that I encountered this error and determined the cause to be a space in the directory name.

In this case:

/Users/james/Development/Personal Projects/

I had a similar situation (storing files in Google Drive which uses /My Drive/). Moving the project to a directory without spaces resolved the issue.

My guess is that nyc isn't properly quoting the directory that's supplied to it as an argument.

jonadeline commented 1 year ago

Same issue here, tried to add nyc as dev dependency but that doesn't bring anything. No spaces in my directories.

image
neilpoulin commented 1 year ago

Plus 1 on this issue. StoryBook runs with no errors, and tests pass with no errors. It looks like code coverage is created successfully (in .nyc_output/coverage.json) and if i run nyc report i see the coverage in the terminal as expected. it just doesn't show up after running npm run test-storybook --coverage.

MisterCommit commented 11 months ago

"storybook:test": "test-storybook --coverage --coverageDirectory .storybook && nyc report --reporter=text -t coverage/storybook --report-dir .storybook",

Visible-Radio commented 7 months ago

I followed this: https://storybook.js.org/tutorials/ui-testing-handbook/react/en/introduction/

and attempted to add the coverage addon according to this: https://storybook.js.org/docs/writing-tests/test-coverage#code-instrumentation-with-the-coverage-addon

And I do not see a coverage report in the terminal when invoking npm run test-storybook --coverage. Nor do I see an nyc output file generated at .nyc_output/coverage.json

Changing the test script to match the suggestion from @MisterCommit does output a report in the terminal - but it is missing many of the files that should have been exercised.

All tests run and pass.

repo here: https://github.com/Visible-Radio/ui-testing-guide-code

vladsovetov commented 5 months ago

Also, experience this issue. Added a quick nextjs repo where you can see that there is no coverage report in the console running pnpm run test-storybook --coverage But it producing coverage/storybook/coverage-storybook.json which can be placed into .nyc_output folder to be able to get the report running nyc report command

edwardscull commented 5 months ago

Also seeing this problem as well some additional issues:

Environment

I followed the Storbook docs https://storybook.js.org/docs/writing-tests/test-coverage and am currently generating a single json file. Coverage file (5797 bytes) written to .nyc_output/coverage.json in the terminal.

I have tried many commands and solutions posted from various sites etc. For example, the command posted above test-storybook --coverage --coverageDirectory .storybook && nyc report --reporter=text -t coverage/storybook --report-dir .storybook, simply doesn't work for me.

Goal

Progress

The only progress I have made was to manually install istanbul and run a command to convert the JSON file into an HTML report. This is when I noticed only 2 of my components/stories were being scanned.

Any ideas would be much appreciated

yannbf commented 5 months ago

Hey everyone! Sorry about this, there is a regression in the Storybook test-runner which causes the report not to be shown. The issue will be fixed in upcoming versions!

yannbf commented 5 months ago

Hey everyone, the issue is now fixed in 0.18.0, closed by #456. Thanks!

yannbf commented 5 months ago

@edwardscull I appreciate the detail you put in your comment!

  • I wold like to generate HTML reports and not just a single JSON file
  • I would like to see the terminal printout showing the results

These are default in 0.18.0

I would like to see all my components in the coverage results. Currently I am only seeing 2 of my components

You might need to use the istanbul options in the addon coverage to include different directories: https://github.com/storybookjs/addon-coverage#configuring-the-addon