mxschmitt / playwright-test-coverage

Playwright Test (@playwright/test) demo to collect coverage information via Istanbul
MIT License
117 stars 16 forks source link

nyc report produces no output #5

Closed ghost closed 2 years ago

ghost commented 2 years ago

I've implemented your steps in a Vue app which we previously had working with Cypress and their code coverage implementation. When I run my playwright tests I see a .nyc_output folder show up with files containing stuff like this:

{"/cosmos/plugins/cosmosc2-tool-base/src/components/ClockFooter.vue":{"path":"/cosmos/plugins/cosmosc2-tool-base/src/components/ClockFooter.vue","statementMap":{"0":{"start":{"line":80,"column":4},"end":{"line":89,"column":5}},...

However when I type yarn nyc report I get:

yarn run v1.22.15
$ C:\COSMOS\playwright\node_modules\.bin\nyc report
----------|---------|----------|---------|---------|-------------------
File      | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s
----------|---------|----------|---------|---------|-------------------
All files |       0 |        0 |       0 |       0 |
----------|---------|----------|---------|---------|-------------------
Done in 0.98s.

I know when we were running with Cypress we had to translate our paths to match up with our source files but it doesn't even appear like I'm getting that far. I'm not running my server via nyc but again I'm pretty sure I'm instrumenting my source files (it worked in our cypress config) and I'm getting output in .nyc_output but I just can't seem to get a good report.

dwalleck commented 2 years ago

I'm seeing the same thing. I'm also noticing the way the Playwright documentation implemented coverage looks different than this example. I'm assuming the approaches have the same outcome, but different ways to start and stop coverage.

ghost commented 2 years ago

I was able to figure out that the --cwd option was required for it to work. I have the following in my package.json:

  "scripts": {
    "coverage": "nyc report --cwd . --reporter=html",
    "clean": "rm -rf .nyc_output || rmdir /s /q .nyc_output; rm -rf coverage || rmdir /s /q coverage; rm -rf test-results || rmdir /s /q test-results"
  },

I then noticed that while I got coverage I was unable to get the correct links back to my source files. This could be due to our use of single-spa or how we've structured our repo. In any case I had to override the node_modules/istanbul-lib-report/lib/context.js code and reimplement function defaultSourceLookup(filePath) to modify the filepath to match what my structure looked like.

ghost commented 2 years ago

If you're interested my full playwright setup is here: https://github.com/BallAerospace/COSMOS/tree/master/playwright

alimtunc commented 2 years ago

Got same issue and this worked for me : npx nyc report --reporter=text --exclude-after-remap false

source : https://github.com/istanbuljs/nyc/issues/1394

ghost commented 2 years ago

Found this blurb somewhere: "Important: If you are using nyc with a project that pre-instruments its code, run nyc with the configuration option --exclude-after-remap set to false. Otherwise nyc's reports will exclude any files that source-maps remap to folders covered under exclude rules."

I'm using webpack 5, babel and the babel-plugin-istanbul to run my Vue application so I assume that means I'm pre-instrumenting my code which requires the --exclude-after-remap option. What's weird is that the --cwd . option also seemed to produce output.

saewitz commented 11 months ago

@mxschmitt I needed this for my Vite project, did you need it as well? It may be good to add to the readme.