search

vuejs / vue-test-utils

Component Test Utils for Vue 2
https://vue-test-utils.vuejs.org
MIT License
3.57k stars 669 forks source link

Tests are much slower since 22 #892

Open trollepierre opened 6 years ago

trollepierre commented 6 years ago

Version

1.0.0-beta.22

Steps to reproduce

VueTestUtils + Jest + snapsnots are slower since beta 22

What is expected?

My tests use to last 70 seconds (900 tests, 70 snapshots) during VTU beta 20.

What is actually happening?

Now they last 170 seconds (900 tests, 70 snapshots) during VTU beta 22

I don't know why it is so slow, but I believe this should be fastened

eddyerburgh commented 6 years ago

Yes we should make sure tests are as fast as possible.

Are you able to provide me a repo that demonstrates the slower tests? That way I can investigate perf improvements.

trollepierre commented 6 years ago

Sorry I am not allowed. Maybe I will create a personal project to help.

trollepierre commented 6 years ago

We have inconsistence when passing tests. (some simple tests needs 8seconds) Sometimes all tests on our pipeline lasts more than 1 hour Sometimes all tests on our pipeline lasts 170-250 seconds... I don't know what happen but I think we will go back to beta 20.

trollepierre commented 6 years ago

This is one test that lasts 8 min on our pipeline https://codesandbox.io/s/308vv8l08p

trollepierre commented 6 years ago

Long tests are fixed in beta 24. I don't know why. Thanks for that. But I stil have test times that is multiplied by 2. I will investigate

trollepierre commented 6 years ago

After investigation, I notice that Jest RAM consumption has exploded from beta 20 to beta 24. That provokes long tests, timeout on my Docker, ... Having a repository with 1000 tests (mixing mount and shallowMount) will be enough to have a good repro (I did it)

trollepierre commented 6 years ago

On that branch, https://github.com/trollepierre/recontact_travel_blog/tree/vue-test-utils-perf

Tests last 18 seconds on vue-test-utils@^beta 20 and 32 seconds on vue-test-utils@^beta 24

The issue is on that component: https://github.com/trollepierre/recontact_travel_blog/blob/vue-test-utils-perf/client/src/components/DTP/DatetimePicker.spec.js (I intentionally duplicate tests for this issue)

eddyerburgh commented 6 years ago

Thanks for the reproduction.

I'm failing to replicate your results on my machine, I actually get a shorter time on beta.24 (~10s) compared to beta.20 (~13s).

I've been profiling the vue-test-utils tests and can't see any obvious expensive function calls. I'll keep investigating, because it's important that Vue Test Utils is fast.

trollepierre commented 6 years ago

Could you limit your RAM use to 2 Go?

We notice a memory leak when executing the tests. Because we can see the RAM consumption increasing on EACH of the Jest process as long as tests are executed.

trollepierre commented 6 years ago

Are you using the good branch? the branch vue-test-utils-perf? On my Mac with 16 Go of RAM, it's 32seconds.

The time of 10 seconds is close to my time without the thousands of duplicated tests...

eddyerburgh commented 6 years ago

Thanks, I switched branches and profiled. The functions that are taking the longest are internal calls to createLocalVue, and external calls to Vue.use.

I'm going to open a PR to minimize internal createLocalVue calls. I'll continue investigating and improving performance.

mkopinsky commented 5 years ago

I don't know if it's related to this, but when running tests with mocha-webpack --watch, we occasionally see tests slow to a crawl, and node sometimes dies entirely with a stack trace/core dump. This points (in my mind) to a memory leak somewhere, but I have no clue if it's in mocha, mocha-webpack, vue-test-utils, or elsewhere.

I'd be happy to post the stack trace or do some other investigation next time this happens - if you have any debugging/profiling recommendations, I would be appreciative!

BrianBrenner commented 5 years ago

I haven't been noticing tests slowing down, but I only have 300 tests so it might not be noticiable yet, but lately the node heap has been running out of memory. I'm running the tests with using Karma and mocha and this plugin for Vue CLI 3 (https://www.npmjs.com/package/vue-cli-plugin-unit-karma). When running with the watch flag the heap runs out of memory after running two or three times.

trollepierre commented 5 years ago

You just need to duplicate 4-5 times your tests (especially the one with mounting options and snapshots...) to replicate and see the difference...

Personally I increased the RAM level on my Gitlab And it works. Le mer. 28 nov. 2018 à 18:14, Brian Brenner notifications@github.com a écrit :

I haven't been noticing tests slowing down, but I only have 300 tests so it might not be noticiable yet, but lately the node heap has been running out of memory. I'm running the tests with using Karma and mocha and this plugin for Vue CLI 3 ( https://www.npmjs.com/package/vue-cli-plugin-unit-karma). When running with the watch flag the heap runs out of memory after running two or three times.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/vuejs/vue-test-utils/issues/892#issuecomment-442528196, or mute the thread https://github.com/notifications/unsubscribe-auth/AH-dRlFYXWd1ygZHgR8q0WhV6RkGM4Moks5uzsRwgaJpZM4VslAS .

--

Pierre Trollé