[Tracking] Coverage prototype

valentinpalkovic commented 1 month ago

RFC

https://github.com/storybookjs/storybook/discussions/28397

Milestones

Use this space to document how the project will be split up. At project kickoff, at minimum we should have all the milestones split up, and emphasize which is must-have and which is nice-to-have. As we dig in, flesh out the detailed implementational tasks in each milestone.

### Milestone 1: Spike
- [ ] https://github.com/storybookjs/storybook/issues/28628
- [ ] https://github.com/storybookjs/storybook/issues/28629
- [ ] https://github.com/storybookjs/storybook/issues/28631
- [x] Spawn Vitest in a subprocess (https://github.com/storybookjs/storybook/pull/28937)

### Milestone 2: Benchmarking
- [x] Benchmark istanbul vs v8
- [x] Benchmark happy-dom vs browser mode
- [x] Benchmark "Component Coverage" vs "Project Coverage"
- [ ] Test: Coverage Differences between @storybook/test-runner and vitest

### Milestone 3: Stretch goals
- [ ] https://github.com/storybookjs/storybook/issues/28630
- [ ] Investigation: CSS Coverage
- [ ] Pre-Build Coverage Report Generation

Nice to Have

### Related issues that may be resolved by this project
- [ ] https://github.com/storybookjs/storybook/issues/28949
- [ ] [Bug]: Failed to fetch dynamically imported module: http://localhost:3000/@id/@vitest/coverage-istanbul?import

Decision Records

Vocabulary

Project Coverage: Comprehensive coverage of the component, including all tests from various stories and higher-level components that include the targeted component.
Story Coverage: Coverage resulting only from the currently selected story (not all stories in one story file)
Component Coverage: Coverage resulting from all stories of the currently selected story file

greptile-apps[bot] commented 1 month ago

Disclaimer This information might be inaccurate, due to it being generated automatically Please provide a valid reproduction link in the 'Reproduction link' section. Refer to the bug report template for guidance. If the link is invalid or missing, the issue may be closed. For more details, see the invalid link comment.

References

/.github/ISSUE_TEMPLATE/bug_report.yml /.github/comments/invalid-link.md /.github/workflows/stale.yml /.github/workflows/triage.yml

#### About Greptile

This response provides a starting point for your research, not a precise solution. Help us improve! Please leave a 👍 if this is helpful and 👎 if it is irrelevant. [Ask Greptile](https://app.greptile.com/chat/github/storybookjs/storybook/next) · [Edit Issue Bot Settings](https://app.greptile.com/apps/github)

cenfun commented 1 month ago

I created a tool MCR that can generate coverage reports directly from V8. It does not require instrumenting source code like Istanbul.

Preview reports https://cenfun.github.io/monocart-coverage-reports/
CSS Coverage support https://cenfun.github.io/monocart-coverage-reports/css/index.html
Example for Storybook https://github.com/cenfun/storybook-monocart-coverage

Not sure if it's useful but maybe an additional option.

storybookjs / storybook