vitest-dev / vitest

Next generation testing framework powered by Vite.
https://vitest.dev
MIT License
12.7k stars 1.14k forks source link

Vitest has too much memory usage in the face of a large number of benches #6539

Open D-Sketon opened 4 days ago

D-Sketon commented 4 days ago

Describe the bug

ref: https://github.com/vitest-dev/vitest/discussions/6535

I'm a contributor of es-toolkit. When I try to run about 200+ bench.ts using yarn vitest bench, the memory footprint is too high. I allocated 26000mb of memory for the node and it still crashes. image benchmark: https://github.com/toss/es-toolkit/tree/main/benchmarks/performance

Reproduction

System Info

System:
    OS: Windows 11 10.0.22631
    CPU: (16) x64 13th Gen Intel(R) Core(TM) i5-13400F
    Memory: 20.48 GB / 31.82 GB
  Binaries:
    Node: 20.13.1 - D:\nodejs\node.EXE
    Yarn: 4.2.2 - D:\nodejs\yarn.CMD
    npm: 10.5.2 - D:\nodejs\npm.CMD
    bun: 1.1.27 - D:\nodejs\bun.CMD
  Browsers:
    Edge: Chromium (127.0.2651.74)
    Internet Explorer: 11.0.22621.3527
  npmPackages:
    @vitest/coverage-istanbul: ^1.5.2 => 1.5.2
    vitest: ^1.5.2 => 1.5.2

Used Package Manager

yarn

Validations

hi-ogawa commented 3 days ago

Since the default reporter doesn't need entire samples array, we can probably free it early and only keep track of samplesCount instead.

Reproduction by generating bench files https://stackblitz.com/edit/vitest-dev-vitest-7ovbya?file=src%2Fgen.mjs