search

nuxt / test-utils

🧪 Test utilities for Nuxt
http://nuxt.com/docs/getting-started/testing
MIT License
322 stars 83 forks source link

Problem with nuxt imports: Vitest Coverage stays at 100% for components regardless of the existence of any component test because of nuxt imports #891

Open FlorianPhilipp2007 opened 3 months ago

FlorianPhilipp2007 commented 3 months ago

Environment

Reproduction

https://stackblitz.com/edit/nuxt-starter-gvfsue?file=nuxt.config.ts,components%2Fnested%2FTestNestedUntested.vue

vitest --coverage is not working in StackBlitz. Please download the project and try it locally.

Describe the bug

Description: When the components option in the Nuxt configuration is set to true (enabling automatic global registration of components), the test coverage report shows 100% coverage for all components, regardless of whether tests actually exist for them. Something is importing the untested TestNestedUntested. V8 is reporting coverage for it. Maybe Nuxt imports all those files.

coverage_bug

Actual Behavior: The coverage report incorrectly shows 100% coverage for all components, including those without any tests like TestNestedUntested.vue in component directory. Seems to be a problem caused by nuxt importing all files.

Expected Behavior: The coverage report should accurately reflect the coverage for each component. Only TestNested.vue should show 100% coverage, while TestNestedUnested.vue should show 0% coverage or appropriate coverage based on the absence of tests.

Additional context

No response

Logs

No response

stackblitz[bot] commented 3 months ago

Fix this issue in StackBlitz Codeflow Start a new pull request in StackBlitz Codeflow.

danielroe commented 3 months ago

This is unlikely something to fix in Nuxt. It might possibly be a vitest bug or something to be resolved in nuxt/test-utils. Have you tried ensuring that .nuxt is added to your coverage exclude patterns?

dosubot[bot] commented 3 months ago

Hey @FlorianPhilipp2007! I'm here to help you with any bugs, questions, or contributions. Let's work together to find a solution!

To resolve the issue where Nuxt imports are causing Vitest coverage reports to incorrectly show 100% coverage for all components, you need to adjust the coverage configuration in your vitest.config.ts file. Specifically, you should ensure that the coverage configuration accurately reflects the files and directories you want to include and exclude from the coverage report.

Here's an updated version of your vitest.config.ts file with a more precise coverage configuration:

import { resolve } from 'pathe'
import { configDefaults, coverageConfigDefaults, defineConfig } from 'vitest/config'
import { isWindows } from 'std-env'

export default defineConfig({
  resolve: {
    alias: {
      '#build/nuxt.config.mjs': resolve('./test/mocks/nuxt-config'),
      '#internal/nuxt/paths': resolve('./test/mocks/paths'),
      '#build/app.config.mjs': resolve('./test/mocks/app-config'),
      '#app': resolve('./packages/nuxt/dist/app'),
    },
  },
  test: {
    globalSetup: './test/setup.ts',
    setupFiles: ['./test/setup-env.ts'],
    coverage: {
      exclude: [...coverageConfigDefaults.exclude, 'packages/nuxt/src/app', 'playground', '**/test/', 'scripts', 'vitest.nuxt.config.ts'],
    },
    testTimeout: isWindows ? 60000 : 10000,
    exclude: [...configDefaults.exclude, 'nuxt/**', '**/test.ts', '**/this-should-not-load.spec.js'],
    poolOptions: {
      threads: {
        maxThreads: process.env.TEST_ENV === 'dev' ? 1 : undefined,
        minThreads: process.env.TEST_ENV === 'dev' ? 1 : undefined,
      },
    },
  },
})

In this configuration:

This should help ensure that the coverage report accurately reflects the tested and untested components [1][2].

To continue talking to Dosu, mention @dosu.

FlorianPhilipp2007 commented 3 months ago

This is unlikely something to fix in Nuxt. It might possibly be a vitest bug or something to be resolved in nuxt/test-utils. Have you tried ensuring that .nuxt is added to your coverage exclude patterns?

I first raised the issue at vitest but they think its a nuxt problem: https://github.com/vitest-dev/vitest/issues/6029#issuecomment-2210914420

I tried the exclude from .nuxt but it doesen't help

danielroe commented 3 months ago

It seems like the issue is the one Ari linked - it's a Vue sourcemap bug: https://github.com/vitest-dev/vitest/issues/3607#issuecomment-1608954728.

AriPerkkio commented 3 months ago

This report contains two issues:

  1. Something (nuxt ?) is importing files that are not being tested. Node's V8 coverage includes every file that is loaded via Node. The repro is testing TestNested.vue, but is also unexpectedly seeing coverage for UntestNested.vue. Vitest is not importing UntestNested.vue.
  2. Coverage is incorrectly 100%. This is caused by Vue's sourcemaps being off. We've seen so many of these issues that I think I'll need to raise an issue on Vue some day.
DenisLug commented 2 months ago

@AriPerkkio, @danielroe: Thank you both for your help so far.

The issue with the coverage report has evolved after updating vitest to version 2 (see comment in vitest issue, Stackblitz demo). Previously, the coverage incorrectly showed 100% for all components, but now it shows 0% coverage for files that do have tests.

To summarize the current status:

Given these changes, could we clarify who will be investigating this further? Is this something that should be addressed by the Nuxt team, or should we continue the discussion with the Vitest team?