Seeking Guidance: Significant performance regression within editor when upgrading typescript versions in large monorepo

mitch1995 commented 1 month ago

Acknowledgement

[x] I acknowledge that issues using this template may be closed without further explanation at the maintainer's discretion.

Comment

Context

Upon upgrading from typescript version 5.4.2 to 5.5.2 we are seeing a large performance regression within our local editor (vscode). We are seeking guidance on what might be causing this regression and how we can find a solution to allow us to proceed with the upgrade. We are not seeing any performance impact when running the actual typescript compilation build (e.g. running tsc -b project-references.tsconfig.json).

We have tested this in multiple versions, such as 5.6.0 and 5.7.0, and are still seeing the same regression. I have also tested typescript version 5.4.5 and have not experienced any regressions, which makes me believe the regression occurred somewhere between 5.4.5 and 5.5.2.

Typescript settings

We are using the same typescript settings mentioned in a previous ticket that we opened (https://github.com/microsoft/TypeScript/issues/59780).

Regression

We are seeing this regression locally within our IDEs (we predominately use vscode). Our monorepo has many sub-packages that are separate TS projects. We have tooling in place that randomly selects and tests the elapsed time it takes a TS project to show type definitions (by looking at the updateOpen elapsed time). For simplicity's sake, I will focus on a single TS project where we are seeing a large regression.

Typescript Version 5.4.2	Typescript Version 5.5.2	Percentage Increase
74.29ms	136.64ms	+84%

updateOpen: elapsed time (in milliseconds)

Typescript Version 5.4.2 updateOpen: elapsed time (in milliseconds) (mean average): 74.29.

Running tsc -p projectDir/tsconfig.json --extendedDiagnostics produces:

Files:                        28110
Lines of Library:             38866
Lines of Definitions:        814186
Lines of TypeScript:        1834871
Lines of JavaScript:              0
Lines of JSON:                 4507
Lines of Other:                   0
Identifiers:                2834483
Symbols:                    7353123
Types:                      2044340
Instantiations:            14039604
Memory used:               6912809K
Assignability cache size:   2807267
Identity cache size:         844956
Subtype cache size:          158977
Strict subtype cache size:   235064
I/O Read time:               10.16s
Parse time:                   4.05s
ResolveModule time:           8.68s
ResolveTypeReference time:    0.02s
ResolveLibrary time:          0.01s
Program time:                24.66s
Bind time:                    2.32s
Check time:                 116.29s
printTime time:               0.73s
Emit time:                    0.84s
transformTime time:           0.06s
commentTime time:             0.00s
I/O Write time:               0.02s
Total time:                 144.11s

The below are snippets taken from the TS Server logs in vscode:

Info 0     [13:51:34.806] Starting TS Server
Info 40    [13:51:34.941] Starting updateGraphWorker: Project: projectDir/tsconfig.json
Info 54126 [13:52:25.108] Finishing updateGraphWorker: Project: projectDir/tsconfig.json projectStateVersion: 1 projectProgramVersion: 0 structureChanged: true structureIsReused:: Not Elapsed: 50167.559082984924ms
Info 54197 [13:52:28.218] Starting updateGraphWorker: Project: projectDir/tsconfig.json
Info 83985[13:52:48.329] Finishing updateGraphWorker: Project: projectDir/tsconfig.json projectStateVersion: 1 projectProgramVersion: 0 structureChanged: true structureIsReused:: Not Elapsed: 20111.59362500906ms
Perf 83993[13:52:51.013] 2::updateOpen: elapsed time (in milliseconds) 76179.5956

The below screenshot is from running tsc -p projectDir/tsconfig.json --generateTrace

Typescript Version 5.5.2 updateOpen: elapsed time (in milliseconds) (mean average): 136.64

Running tsc -p projectDir/tsconfig.json --extendedDiagnostics produces:

Files:                        28246
Lines of Library:             39135
Lines of Definitions:        820870
Lines of TypeScript:        1834871
Lines of JavaScript:              0
Lines of JSON:                 4507
Lines of Other:                   0
Identifiers:                2843999
Symbols:                    7100120
Types:                      1952998
Instantiations:            15389105
Memory used:               6624656K
Assignability cache size:   2762165
Identity cache size:         845019
Subtype cache size:          158941
Strict subtype cache size:   217035
I/O Read time:               11.47s
Parse time:                   7.54s
ResolveModule time:          12.34s
ResolveTypeReference time:    0.02s
ResolveLibrary time:          0.02s
Program time:                34.61s
Bind time:                    3.01s
Check time:                 108.32s
I/O Write time:               0.01s
printTime time:               0.49s
Emit time:                    0.49s
Total time:                 146.43s

The below are snippets taken from the TS Server logs in vscode:

Info 0    [14:06:01.540] Starting TS Server
Info 48   [14:06:01.705] Starting updateGraphWorker: Project: projectDir/tsconfig.json
Info 12837[14:07:14.662] Finishing updateGraphWorker: Project: projectDir/tsconfig.json projectStateVersion: 1 projectProgramVersion: 0 structureChanged: true structureIsReused:: Not Elapsed: 72938.59612500668ms
Info 12850[14:07:17.960] Starting updateGraphWorker: Project: projectDir/tsconfig.json
Info 18115[14:08:07.940] Finishing updateGraphWorker: Project: projectDir/tsconfig.json projectStateVersion: 1 projectProgramVersion: 0 structureChanged: true structureIsReused:: Not Elapsed: 49968.97329199314ms
Perf 18116[14:08:10.834] 2::updateOpen: elapsed time (in milliseconds) 129234.7819

The below screenshot is from running tsc -p projectDir/tsconfig.json --generateTrace

If you require any additional info, please let me know and I will work on getting it for you

Andarist commented 1 month ago

If you could bisect this to the exact nightly version that introduced this perf degradation that would certainly help to start investigating this

mitch1995 commented 1 month ago

Hey, sorry for the late reply. I was out of the office for a day or so. I've just tested the nightly builds and can see that the regression first occurred in 5.5.0-dev.20240413. I've included some relevant data below to support my belief that the regression first began occurring in 5.5.0-dev.20240413

Typescript version 5.4.5 (updateOpen elapsed time over 20 executions):
44.95,
45.11,
48.01,
49.69,
50.01,
50.32,
50.36,
50.75,
50.93,
51.02,
50.51,
52.35,
49.47,
50.23,
48.62,
49.97,
49.59,
49.88,
48.8,
51.15

Typescript version 5.5.0-dev.20240412 (updateOpen elapsed time over 20 executions):
44.99,
51.89,
49.94,
51.72,
46.36,
44.98,
46.94,
51.4,
48.83,
50.07,
51.24,
50.64,
50.97,
50.12,
51.11,
44.05,
49.72,
49.41,
51.41,
45.27

Typescript version 5.5.0-dev.20240413 (updateOpen elapsed time over 20 executions):
64.79,
68.97,
75.01,
75.82,
72.53,
75.97,
68.51,
75.75,
67.68,
76.54,
69.29,
68.62,
67.1,
68.12,
68.29,
69.37,
67.76,
67.58,
68.15,
69.16

Typescript version 5.5.2 (updateOpen elapsed time over 20 executions):
65.55,
76.3,
70.51,
75.43,
65.83,
74.28,
70.77,
75.13,
68.04,
67.39,
67.07,
67.58,
67.29,
67.31,
66.89,
67.95,
66.82,
67.35

mitch1995 commented 1 month ago

I tested the individual commits within that nightly build and can see that the regression began in https://github.com/microsoft/TypeScript/pull/58139. Below are the updateOpen elapsed times for the two merge commits

commit: 6092c2d4c40469a2faa1e4775f0f3a47e023c176
61117.2838
61342.6039
61782.7225
60921.6084
61119.6564

commit: b0067685487068e049533709221f4341e96f3f21
76354.6758
76365.8160
76600.0248
77000.1363
76978.6095

cc @sheetalkamat - tagging you since you are the PR author - would you happen to have any idea on why this regression has occurred and any possible ideas on how we can overcome it?

ReDrUm commented 4 weeks ago

Note that we do have symlinks in place in our codebase, so the perf regression is probably the overhead of now following those symlinks and adding extra listeners per workspace. If anyone has a solution to avoid the overhead in the meantime (other than removing symlinks), we'd love to hear it.

microsoft / TypeScript