search

sublimehq / sublime_text

Issue tracker for Sublime Text
https://www.sublimetext.com
804 stars 39 forks source link

Apply syntax highlighting to views asynchronously #2603

Closed kylebebak closed 5 years ago

kylebebak commented 5 years ago

Problem description

Syntax highlighting is applied synchronously to views when they're opened, or when syntax is applied to files being previewed in the Goto File overlay.

For larger files for which setting syntax takes a while, this causes significant lag when opening new files, tabbing through files, or previewing files.

Here's an example of opening a ~600-line JS file. The editor hangs for about a second while syntax is set on the view.

out

Preferred solution

ST could handle this the same way editors like Atom handle it, namely, it could set the syntax on the view asynchronously.

It could push this job onto another thread that doesn't block the UI thread. It would initially show the view text without any highlighting, and highlight it once this job is finished.

Alternatives

Really none that I can think of. If setting view syntax blocks the UI thread, there's guaranteed lag with a number of key features, and the bigger the file the more severe the lag. It makes ST, which is otherwise is super fast, feel quite slow at times.

Thom1729 commented 5 years ago

What the heck kind of syntax definition is taking a full second to highlight JavaScr—

JS Custom – React

Oh. Hmm. Is this typical of syntax highlighting performance on your machine, or is this file an outlier for some reason? I feel that it shouldn't take that long for a mere 600 lines of code. Everything should be using Sublime's custom regexp engine, so a pathological regexp shouldn't be to blame.

wbond commented 5 years ago

Yeah, 600ms should handle about 60k lines of C++, so something is off here. Fixing that would seem to be the correct approach.

@kylebebak Actually not highlighting the file asynchronously is one of my favorite things about Sublime Text over other editors. Nothing is jankier than opening a 400 line Python file and watching a modern core i7 scan through the identifiers adding color.

FYI, we don’t use syntaxes just for color, but also tokenization. Changing that to be async would probably mean completely reimplementing the core editor and the API, and would likely make things slower since we’d need a synchronization technique to make sure tokenizing in a background thread along with user changes and API updates didn’t stomp on each other.

FichteFoll commented 5 years ago

(As a side note, I would probably make the non-highlighted version read-only until the background tokenizer finished. That sounds vastly more simple than adding a lot of code for synchronization.)

Thom1729 commented 5 years ago

@kylebebak Can you post your JS Custom configuration? I wonder if some combination of options is adding a pathological combination of lookaheads.

kylebebak commented 5 years ago

@wbond It sounds like this would be a real complicated change with a number of trade-offs, even if it was implemented the @FichteFoll suggested.

It's probably worth mentioning I don't see significant lag when syntax is set on Python or Go files of similar sizes, just JS files.

@Thom1729 I don't have a JS Custom.sublime-settings in my User directory -- I'm using the default settings. I've noticed this slowness with syntax highlighting of JS files for at least a few months.

Here's a video of a different project, using the Goto File overlay. I'm holding the up arrow key and ST is previewing a bunch of JS (React) files, none of which is more than 500 lines long.

https://streamable.com/hw1hb

There's noticeable lag for setting syntax on most of the files, even ones with 150 LOC.

Thom1729 commented 5 years ago

It shouldn't be doing that. I'll look into this over the weekend. It's possible that a performance regression snuck in at some point.

mechatroner commented 5 years ago

This issue is especially badly affecting csv files highlighting and my rainbow_csv extension see issue https://github.com/mechatroner/sublime_rainbow_csv/issues/9 Please fix this!

FichteFoll commented 5 years ago

@mechatroner I imagine your package would benefit greatly from utilizing the new sublime-syntax format and pushing contexts when a separator character is encountered instead of using a huge regex (in complexity) to parse each line column and make this a non-issue, unless you are editing files with hundreds of megabytes.

kylebebak commented 5 years ago

I think this issue should be closed.

I created it because syntax highlighting for my JS files is taking too long, but I made a mistake creating it in this repo. I thought it was related to a perf issue in Sublime Text, but it's much more likely it's a perf issue in JS Custom.

As Will mentioned, changing this behavior would be very costly, and for syntaxes for which syntax highlighting is already very fast (which seems to be most of them), highlighting asynchronously wouldn't help.

kylebebak commented 5 years ago

I'm REALLY sorry for creating this issue.

I found an unmaintained package called ColorHighlighter was responsible for the massive slowdown in highlighting JS syntax.

I removed this package and highlighting JS files is now as fast as it used to be.

FichteFoll commented 5 years ago

As a suggestion for a replacement, I've found ColorHelper to be universally better than ColorHighlighter in all regards.

kylebebak commented 5 years ago

@FichteFoll

Thanks for the suggestion, this looks excellent.

On a different note, it can be easy to miss packages like ColorHelper, because they were created after the Sublime Text "boom times". This often means (sometimes vastly) inferior packages created earlier have (sometimes vastly) more downloads, stars, and visibility.

FichteFoll commented 5 years ago

Yes, that's hard to fix. Especially old "must have" package lists that never get updated are a prime reason why some old and effectively deprecated packages still get a lot of installations.