cenfun / monocart-coverage-reports

A code coverage tool to generate native V8 reports or Istanbul reports.
MIT License
69 stars 6 forks source link

[Bug] Wrong calculation of branch coverage in `switch/case` #68

Closed stereobooster closed 2 months ago

stereobooster commented 2 months ago

Describe the bug

I have switch/case with 3 branches and all covered, but coverage says 2/3

To Reproduce

Expected behavior

Coverage 3/3

Errors or Screenshots

n/a

Make a minimal reproduction

export function experiment(a) {
  let result = 0
  switch (a) {
    case 1:
      result++;
    case 3:
      result++;
    default:
      result++;
  }
  return result;
}

Test

test("experiment", () => {
  expect(experiment(1)).toBe(3);
  expect(experiment(2)).toBe(1);
  expect(experiment(3)).toBe(2);
});

Additional context

n/a

cenfun commented 2 months ago

Yes, it is a bug, but I'm not sure if it can be fixed. The reason is that the V8 coverage data does not provide enough information to calculate the branch coverage. for example, here is V8 coverage ranges:

{
    "functionName": "experiment",
    "ranges": [
        {
            "startOffset": 185,
            "endOffset": 360,
            "count": 3
        },
        {
            "startOffset": 249,
            "endOffset": 273,
            "count": 1
        },
        {
            "startOffset": 279,
            "endOffset": 303,
            "count": 2
        }
    ],
    "isBlockCoverage": true
}

And by parsing code AST, we can get a total of 3 branches, but whether each branch is covered is temporarily determined by simply calculating the count. switch default count = total count 3 - case 1 count 1 - case 3 count 2 = 0 (this is wrong calculation becuase there is no break for switch/case) For normal case switch/case should be:

export function experiment(a) {
  let result = 0
  switch (a) {
    case 1:
      result++;
      break;  <==============
    case 3:
      result++;
      break;  <==============
    default:
      result++;
  }
  return result;
}

Maybe we can calculate if there is a break in branch. Welcome to provide a better solution.

stereobooster commented 2 months ago

Hi. Thank you for fast response. And thank you for maintaining the project. By no means I meant to offend you or say that this is somehow a bad project. You did an impressive job to push poor v8 coverage API to it's limits.

I just noticed one bug, and I thought I would report it. I can't say this comes from real project. I found it by poking around. Feel free to close.

cenfun commented 2 months ago

Thank you for reporting this issue. I think we can keep it open. Maybe one day there is a solution come up.

cenfun commented 2 months ago

@stereobooster It should be fixed, please try monocart-coverage-reports@2.10.3

test case file: https://github.com/cenfun/monocart-coverage-reports/blob/main/test/mock/src/branch/switch-no-break.js coverage report: https://cenfun.github.io/monocart-coverage-reports/v8/#page=test/mock/src/branch/switch-no-break.js

stereobooster commented 2 months ago

Thank you for fixing it