Vue SSR performance inconsistency

[zsilbi]

Prerequisites

• [X] I have written a descriptive issue title
• [X] I have searched existing issues to ensure the issue has not already been raised

Issue

Hello!

This is a follow up on the performance problem that I experienced when running the Vue benchmark in https://github.com/platformatic/ssr-performance-showdown

I noticed the poor performance of the Vue renderer in the original version of the benchmark suite when I saw the first results.

Then I ran the benchmark myself and got similar results. (120-140 req/s)

In this pull request I removed the <style> block from the Vue component and the results were immediately got much better. (330-340 req/s)

Later I tested the original version multiple times and noticed that the results are not consistent. See this comment

Then I recreated the same benchmark using another library and could not reproduce the problem. See this comment

Today I had time to run the original benchmark on a slower computer and found out that the problem is there regardless of the presence of <style> block. I also copied the test component to the vue-next example of this repository and the results were exactly the same.

First run:

┌─────────┬───────┬───────┬────────┬────────┬──────────┬──────────┬────────┐
│ Stat    │ 2.5%  │ 50%   │ 97.5%  │ 99%    │ Avg      │ Stdev    │ Max    │
├─────────┼───────┼───────┼────────┼────────┼──────────┼──────────┼────────┤
│ Latency │ 44 ms │ 53 ms │ 353 ms │ 387 ms │ 81.53 ms │ 81.86 ms │ 734 ms │
└─────────┴───────┴───────┴────────┴────────┴──────────┴──────────┴────────┘
┌───────────┬─────────┬─────────┬─────────┬─────────┬─────────┬─────────┬─────────┐
│ Stat      │ 1%      │ 2.5%    │ 50%     │ 97.5%   │ Avg     │ Stdev   │ Min     │
├───────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Req/Sec   │ 26      │ 26      │ 147     │ 175     │ 119.3   │ 61.21   │ 26      │
├───────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Bytes/Sec │ 5.17 MB │ 5.17 MB │ 29.2 MB │ 34.8 MB │ 23.7 MB │ 12.2 MB │ 5.16 MB │
└───────────┴─────────┴─────────┴─────────┴─────────┴─────────┴─────────┴─────────┘

Immediate second run in the same process:

┌─────────┬────────┬────────┬────────┬────────┬──────────┬───────────┬────────┐
│ Stat    │ 2.5%   │ 50%    │ 97.5%  │ 99%    │ Avg      │ Stdev     │ Max    │
├─────────┼────────┼────────┼────────┼────────┼──────────┼───────────┼────────┤
│ Latency │ 246 ms │ 419 ms │ 806 ms │ 878 ms │ 435.6 ms │ 119.76 ms │ 943 ms │
└─────────┴────────┴────────┴────────┴────────┴──────────┴───────────┴────────┘
┌───────────┬────────┬────────┬─────────┬─────────┬─────────┬────────┬────────┐
│ Stat      │ 1%     │ 2.5%   │ 50%     │ 97.5%   │ Avg     │ Stdev  │ Min    │
├───────────┼────────┼────────┼─────────┼─────────┼─────────┼────────┼────────┤
│ Req/Sec   │ 1      │ 1      │ 22      │ 24      │ 20.11   │ 6.54   │ 1      │
├───────────┼────────┼────────┼─────────┼─────────┼─────────┼────────┼────────┤
│ Bytes/Sec │ 199 kB │ 199 kB │ 4.37 MB │ 4.77 MB │ 3.99 MB │ 1.3 MB │ 199 kB │
└───────────┴────────┴────────┴─────────┴─────────┴─────────┴────────┴────────┘

Immediate second run with concurrent 200 connections:

┌─────────┬──────┬──────┬───────┬──────┬──────┬───────┬──────┐
│ Stat    │ 2.5% │ 50%  │ 97.5% │ 99%  │ Avg  │ Stdev │ Max  │
├─────────┼──────┼──────┼───────┼──────┼──────┼───────┼──────┤
│ Latency │ 0 ms │ 0 ms │ 0 ms  │ 0 ms │ 0 ms │ 0 ms  │ 0 ms │
└─────────┴──────┴──────┴───────┴──────┴──────┴───────┴──────┘
┌───────────┬─────┬──────┬─────┬───────┬─────┬───────┬─────┐
│ Stat      │ 1%  │ 2.5% │ 50% │ 97.5% │ Avg │ Stdev │ Min │
├───────────┼─────┼──────┼─────┼───────┼─────┼───────┼─────┤
│ Req/Sec   │ 0   │ 0    │ 0   │ 0     │ 0   │ 0     │ 0   │
├───────────┼─────┼──────┼─────┼───────┼─────┼───────┼─────┤
│ Bytes/Sec │ 0 B │ 0 B  │ 0 B │ 0 B   │ 0 B │ 0 B   │ 0 B │
└───────────┴─────┴──────┴─────┴───────┴─────┴───────┴─────┘

After this a single curl request takes around 5-6 seconds.

If I wait a few seconds between runs the average is similar in both runs (around 140 rps).

[galvez]

I'm unable to reproduce this disparity.

After the consistency fixes Vue already fared better than all the others except fastify-html.

[zsilbi]

Thanks for the quick reply @galvez

This problem is reproducable in this version https://github.com/platformatic/ssr-performance-showdown/commit/41b704b8d07d4d8b29b1fba1fdb3575b41dce17b of the benchmark, I tested on two machines.

[galvez]

I see — it doesn't seem to reproducible in its current state. I'll verify carefully later.