generated project coverage blank with @web/test-runner-puppeteer

[sashafirsov]

Steps to replicate: In Ubuntu Linux(WSL and github CI) without browser.

npm init @open-wc as web component with test enabled. npm i -D @web/test-runner-puppeteer

npm run test passed successfully, but coverage/lcov.info is blank(0 bytes) and coverage/lcov-report/index.html has no data.

Expected: non-empty lcov.info and coverage html.

Same generated component in environment with usual browser generates coverage as should.

Bug is preventing to include test coverage in github CI which meant to use headless puppeteer browser.

[sashafirsov]

https://github.com/modernweb-dev/example-projects/tree/master/puppeteer npm run test also produces blank coverage file when runs in Ubuntu

[skortchmark9]

I'm noticing this as well - pretty unfortunate for our CI workflow.

[citkane]

I am seeing this Sept 2022 on Ubuntu 22.04.

"--playwright --coverage" as a replacement for "--puppeteer --coverage" produces non-empty coverage reports.

It would be nice to use puppeteer with coverage on Ubuntu though...

[Didel]

The issue unfortunately still exists in the current versions when running wtr --coverage --puppeteer. In have updated the related dependencies to their newest versions to no avail:

├── @web/test-runner-puppeteer@0.15.0
├── @web/test-runner@0.18.0

(output of npm ls, which lists the actually installed version of packages)

Both the cobertura and the lcov reporters are empty, so it seems that this is not a reporter-specific issue.

Has someone already found a solution or workaround for this almost 3 year old issue (other than not using puppeteer)?

[Didel]

I think I have found a work-around:

Previously, I was running wtr using wtr --coverage --puppeteer. My web-test-runner.config.js configuration file (see https://modern-web.dev/docs/test-runner/cli-and-configuration/#configuration-file) did not contain a browsers section:

export default ({
  files: "**/*.test.js",
});

(unrelated configuration not shown)

Work-around

What ultimately worked for me was to explicitly import and add the puppeteerLauncher as follows:

• First, remove the puppeteer-flag from the wtr command (the example given above simply becomes wtr --coverage).

• Then, adjust the configuration file: Import the puppeteerLauncher and add it to the browsers section (no custom configuration or launchoptions are needed) :

import { puppeteerLauncher } from '@web/test-runner-puppeteer';

export default ({
  files: "**/*.test.js",
  browsers: [puppeteerLauncher()],
});

I was able to run the tests just like before, while using Puppeteer. Coverage reports (both in lcov and cobertura format) are correctly generated and contain coverage as expected.

This work-around obviously still does not fix the actual bug (wtr should properly work when using the --coverage and --puppeteer flags), but at least provides a work-around for those relying on Puppeteer.