search

microsoft / TypeScript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
https://www.typescriptlang.org
Apache License 2.0
100.46k stars 12.42k forks source link

TS Server performance 10-50mb files #28723

Open mjbvz opened 5 years ago

mjbvz commented 5 years ago

From https://github.com/Microsoft/vscode/issues/63473

TypeScript Version: 3.3.0-dev.20181128

Search Terms:

Repo Open one of the files from the user's project. These are very large files, some with 1.5 million lines

Bug TS Server goes unresponsive. Here's one example log file: tsserver.log

Two observations:

Err 306   [18:52:6.115] Exception on executing command {"seq":12,"type":"request","command":"navtree","arguments":{"file":"/Users/matb/Downloads/www/code/80000.chunk.js"}}:

    No Project.

    Error: No Project.
        at Object.ThrowNoProject (/Users/matb/Downloads/www/node_modules/typescript/lib/tsserver.js:120007:23)
        at IOSession.Session.getFileAndLanguageServiceForSyntacticOperation (/Users/matb/Downloads/www/node_modules/typescript/lib/tsserver.js:125190:42)
        at IOSession.Session.getNavigationTree (/Users/matb/Downloads/www/node_modules/typescript/lib/tsserver.js:125539:31)
        at Session.handlers.ts.createMapFromTemplate._a.(anonymous function) (/Users/matb/Downloads/www/node_modules/typescript/lib/tsserver.js:124455:61)
        at /Users/matb/Downloads/www/node_modules/typescript/lib/tsserver.js:125847:88
        at IOSession.Session.executeWithRequestId (/Users/matb/Downloads/www/node_modules/typescript/lib/tsserver.js:125838:28)
        at IOSession.Session.executeCommand (/Users/matb/Downloads/www/node_modules/typescript/lib/tsserver.js:125847:33)
        at IOSession.Session.onMessage (/Users/matb/Downloads/www/node_modules/typescript/lib/tsserver.js:125869:35)
        at Interface.<anonymous> (/Users/matb/Downloads/www/node_modules/typescript/lib/tsserver.js:127129:27)
        at emitOne (events.js:116:13)
        at Interface.emit (events.js:211:7)
        at Interface._onLine (readline.js:282:10)
        at Interface._normalWrite (readline.js:424:12)
        at Socket.ondata (readline.js:141:10)
        at emitOne (events.js:116:13)
        at Socket.emit (events.js:211:7)
        at addChunk (_stream_readable.js:263:12)
        at readableAddChunk (_stream_readable.js:250:11)
        at Socket.Readable.push (_stream_readable.js:208:10)
        at Pipe.onread (net.js:594:20)
AnyhowStep commented 5 years ago

1.5 million! I'm super curious about what's in those millions of lines

mjbvz commented 5 years ago

I'm investigating why we start making requests on the 80000.chunk.js file even though it is never opened on the TS Server

(Note that the stall seems to happen before this and on a different file)

mjbvz commented 5 years ago

The originally file was 50.2 mb. Seems like a VS code error that caused us to start making requests for this file without opening it first. 50mb is the limit of what we synchronize with extensions. I haven't been able to trigger it again though

The smaller, 10-15mb files in the repo are likely more interesting. Here's the log: tsserver.log.zip

It's seems faster with 3.3 and I don't see any stalls. The long running requests:

sheetalkamat commented 5 years ago

@mjbvz On our side the large file is classified as anything more than 4mb. https://github.com/Microsoft/TypeScript/blob/master/src/server/editorServices.ts#L4

mjbvz commented 5 years ago

Does that mean we should not sync files over that size with TS Server?

sheetalkamat commented 5 years ago

Does that mean we should not sync files over that size with TS Server?

Right.

mjbvz commented 5 years ago

We can do that. On the server side, should requests against large files be treated as noops? Right now it looks like ts server still tries handling them

RyanCavanaugh commented 5 years ago

@mjbvz what's left to do here?

mjbvz commented 5 years ago

I'll test this again in the latest VS Code