Coverage stays at 100% for components regardless of the existence of any component test

FlorianPhilipp2007 commented 3 weeks ago

Describe the bug

Description: When the components option in the Nuxt configuration is set to true (enabling automatic global registration of components), the test coverage report shows 100% coverage for all components, regardless of whether tests actually exist for them.

Actual Behavior: The coverage report incorrectly shows 100% coverage for all components, including those without any tests like TestNestedUntested.vue in component directory

Expected Behavior: The coverage report should accurately reflect the coverage for each component. Only TestNested.vue should show 100% coverage, while TestNestedUnested.vue should show 0% coverage or appropriate coverage based on the absence of tests.

Reproduction

https://stackblitz.com/edit/nuxt-starter-gvfsue?file=nuxt.config.ts,components%2Fnested%2FTestNestedUntested.vue

vitest --coverage is not working in StackBlitz. Please download the project and try it locally.

System Info

Nuxt Version: ^3.12.2
@nuxt/test-utils: 3.12.1
vitest: 1.5.2
@vitest/coverage-v8: 1.5.2
start-server-and-test: 1.15.5
vitest-environment-nuxt: 1.0.0
@vue/test-utils: ^2.4.6
happy-dom: ^14.12.3

Used Package Manager

npm

Validations

[X] Follow our Code of Conduct
[X] Read the Contributing Guidelines.
[X] Read the docs.
[X] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
[X] Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
[X] The provided reproduction is a minimal reproducible example of the bug.

AriPerkkio commented 3 weeks ago

Something is importing the untested TestNestedUntested. V8 is reporting coverage for it. Maybe Nuxt imports all those files?

{
  "scriptId": "745",
  "url": "file:///x/nuxt-starter-gvfsue/components/nested/TestNestedUntested.vue",
  "functions": [
    {
      "functionName": "",
      "ranges": [{ "startOffset": 0, "endOffset": 981, "count": 1 }],
      "isBlockCoverage": true
    },
    {
      "functionName": "",
      "ranges": [{ "startOffset": 13, "endOffset": 981, "count": 1 }],
      "isBlockCoverage": true
    },
    {
      "functionName": "",
      "ranges": [{ "startOffset": 444, "endOffset": 564, "count": 0 }],
      "isBlockCoverage": false
    }
  ]
}

Coverage is also off because source maps from Vue (or Nuxt?) don't look alright: https://evanw.github.io/source-map-visualization/#NzIwAGNvbn.

Looks similar to https://github.com/vitest-dev/vitest/issues/3607. I would recommend to open an issue on Nuxt or Vue.

FlorianPhilipp2007 commented 1 week ago

Something is importing the untested TestNestedUntested. V8 is reporting coverage for it. Maybe Nuxt imports all those files?
{
  "scriptId": "745",
  "url": "file:///x/nuxt-starter-gvfsue/components/nested/TestNestedUntested.vue",
  "functions": [
    {
      "functionName": "",
      "ranges": [{ "startOffset": 0, "endOffset": 981, "count": 1 }],
      "isBlockCoverage": true
    },
    {
      "functionName": "",
      "ranges": [{ "startOffset": 13, "endOffset": 981, "count": 1 }],
      "isBlockCoverage": true
    },
    {
      "functionName": "",
      "ranges": [{ "startOffset": 444, "endOffset": 564, "count": 0 }],
      "isBlockCoverage": false
    }
  ]
}
Coverage is also off because source maps from Vue (or Nuxt?) don't look alright: https://evanw.github.io/source-map-visualization/#NzIwAGNvbn.

Looks similar to #3607. I would recommend to open an issue on Nuxt or Vue.

I raised the problem at nuxt but they think its a vitest bug https://github.com/nuxt/test-utils/issues/891#issuecomment-2230925946

AriPerkkio commented 1 week ago

@FlorianPhilipp2007 could you test if coverage.provider: 'istanbul' works better here? Under the hood it works differently than the default v8 provider. It could help in hiding the untested files, but the report will still be off due to source maps issues.

vitest-dev / vitest