search

atom / find-and-replace

Find and replace in a single buffer and in the project
MIT License
242 stars 197 forks source link

find/replace performance #145

Closed jdavisclark closed 8 years ago

jdavisclark commented 10 years ago

there are some pretty serious issues with find/replace performance. I realize that some of this is an issue with speeding up marker displays in Atom core (and is supposedly already logged in the closed issue tracker), but i wanted to make sure there was a public issue as well.

e. g. stripping newlines in a ~300 line file via Find: "\r|\n" and Replace:"" can easily take over 10 seconds.

benogle commented 10 years ago

For reference, the marker views are currently in this repo: https://github.com/atom/find-and-replace/blob/master/lib/marker-view.coffee At some point, we intend them to go in core so other packages can use them, but for now, they are in here.

The slow parts:

If someone feels the itch, I would start by profiling. Search for a space in a long text.

I feel like this is @nathansobo territory.

screen shot 2014-03-04 at 5 23 18 pm

nathansobo commented 10 years ago

I wonder if we should support maintaining a cursor in the SpanSkipList for big scans like this. That way we wouldn't need to scan from the beginning every single time. It would be nice to see the profile beneath ::totalTo to get a sense of anything simple we could optimize there.

benogle commented 10 years ago

I wonder if the subscription stuff is lowest hanging fruit. I'm surprised all the eventing stuff is in that profile. That anon function above program is also all subscription stuff. Each MarkerView subscribes to 3 events on the marker, and one on the editor. What if someObject (find model?) could subscribe to all events for all search-result markers, then delegate to the marker view?

Also, what's creating all the garbage? I guess a heap profile would be in order. Probably more than just dom elements.

nathansobo commented 10 years ago

Marker views really need to go into core so we can optimize this stuff once.

DiogoAndre commented 10 years ago

I'm working with a 22k LOC CoffeeScript file. Every time I want to search for, lets say, 'cucumber' - quotes included - Atom tries to match the first single quote before I can even type the next characters, and locks completely.

Maybe introduce a slight delay between typing and starting the search?

Well, maybe the real problem here is having 22k lines of CoffeeScript :scream:

benogle commented 10 years ago

There is a delay, it's just not very long (300ms?).

DiogoAndre commented 10 years ago

I couldn't find where the delay is set... Another solution would be to bring back the search after 1st char behavior. It improves the experience when editing large files, and searching for just one char can still be achieved by core:confirm

lifeisfoo commented 10 years ago

Try a regex find in Android WebView.java with /*: Atom freezes.

pabliqe commented 10 years ago

Replacing [tab] characters for double [blank-space] on >3000 matches makes the software to freeze and throw "keep waiting" confirmations like 3 times 'till its done.

dbkaplun commented 10 years ago

I think the best solution (at least as a stopgap) is to offload processing to another process, so at least the UI is unblocked while finding happens.

Fank commented 9 years ago

I also noticed if you searched something with RegEx (e.g. /,(\n+\t*)/, which were very slow, single core 100% usage) and stop searching -> leave search "pattern" in search filed but closes the search view by pressing "Escape", its still slow, because it feels like its still searching the pattern after every content change. So when i open the search view again and disable the RegEx option and empty the search pattern and close it via "Escape" too, same as before. It runes fine/smooth.

Another thing i noticed is if you use the RegEx example about and replace the first line its fast, second slower, third slower as second, after the 7-10 replacement it will take about 10-30 sec to proceed 1 line replacement.

Tested in atom 0.174 on Ubuntu 14.04 (Linux **** 3.13.0-44-generic #73-Ubuntu SMP Tue Dec 16 00:22:43 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux) with a perl module (.pm) about 760 lines

I could test this on windows too if desired

Cipa commented 9 years ago

For this file https://gist.github.com/Cipa/62e40f9ac95d65ddcadd I tried to remove the empty lines with find/replace(I just copied the empty space between 2 lines and tried to replace it with nothing) and currently I am at the 4th Keep Waiting message and it still did not do it. I think it crashed.

Same thing takes 5 seconds or less in Coda.

Atom 0.179.0, started in safe mode

benogle commented 9 years ago

Thanks for the test case. Sorry it's slow :(

Fank commented 9 years ago

Today i replaced a file with 15k lines all step by step (not replace all), and i noticed the first time i opened the file in atom i could nearly instand replace line step by step, after about 200 lines it getting slow (800ms/line). So i closed the file and reopened it (same atom session/window), it the time decreases to ~150ms foreach line, after some replacements (~200) it increases to ~2sec. 3., 4. ... same behaviour as the second opening, same times no time increasing.

I used this RegEx ,(((?:\s+--.*)?\r?\n)+\s+)+ with folling in replacement $1,, only the RegEx option were enabled.

Tested in 0.194.0 on linux same as my comment above, i hope this information help in someway.

willopez commented 9 years ago

This is the reason I don't use Atom as my main text editor. I want to spend time coding, not waiting for a tool. When I get some free time I will look into the possibility of using https://github.com/ggreer/the_silver_searcher as the backend and building an Atom plugin similar to https://github.com/rking/ag.vim

benogle commented 9 years ago

@willopez what specifically is slow for you? Is it search in buffer? In project? What are you searching for?

maletor commented 9 years ago

+1 on the silver searcher.

railrhoad commented 9 years ago

Search in general on a smaller file takes a fair amount of time. Doesn't matter how many files are included in the project. I have a 30kb json file, replacing the 3000 quotes to double quotes took about 45 seconds time to do.

jdavisclark commented 9 years ago

@railrhoad thats the exact type of thing I use it for a lot in other editors

jdavisclark commented 9 years ago

@nathansobo awesome news with the marker performance! cant wait.

nathansobo commented 9 years ago

@jdc0589 It's worth noting that creating markers could still use some optimization. But once they're on screen life should be much better. Stay tuned.

sublines commented 9 years ago

I would like to add another performance issue to the discussion. When you work on several files at once (which we do all the time) and you're in a file and search for something but then switch back to another file without closing the search first, it will automatically also search the file you went back to or any newly opened file. I always leave the search bar open after I've searched for something and of course I'm glad it doesn't close automatically when you change to another file. But please, add a settings function to disable auto searching everytime you switch between files or open a new one.

TimvdLippe commented 9 years ago

My editor also reports to keep on hanging when performing the regex \\n I need to replace all \n with actual endline characters. This regex on a file with 2900 matches found is currently taking a couple of minutes before finishing.

benogle commented 8 years ago

This has been improved significantly via a new marker index which is currently in master. Check out these PRs for more information

As with any performance things, more work can always be done, and we have ideas. We will create more specific issues moving forward.

jdavisclark commented 8 years ago

thanks a bunch @benogle! I've actually been using atom some more recently (as my preferred markdown editor) and have noticed some solid marker improvements.