search

atom / find-and-replace

Find and replace in a single buffer and in the project
MIT License
242 stars 197 forks source link

Find and Replace in large buffer is very slow #653

Open benogle opened 8 years ago

benogle commented 8 years ago

From @ypresto on January 21, 2016 7:59

It seems to rebuilding tokenized buffer on every operation in transaction.

image

It took 16 secs to replace 920 \ns to newline on 15-inch MacBook Pro Retina Late-2013.

Copied from original issue: atom/atom#10531

benogle commented 8 years ago

This is actively being worked on by allowing multiple updates to update the interface and any listeners once. iirc, we do not have an issue to link to at this time. @nathansobo are there any issues related to this?

lllShamanlll commented 8 years ago

Are there any movements here? Didn't find any related atom issue.

nathansobo commented 8 years ago

There was improvement following the issue, but it's always a tricky issue to discuss when performance is "good enough" to close an issue. My answer is that it probably still isn't good enough and we're focusing on some other areas for now. We will revisit find and replace performance in the future.

nathansobo commented 8 years ago

That said, if you'd like to post profile data on any performance issue it is always welcome.

lllShamanlll commented 8 years ago

Actually this looks like atom issue. Due to lack of methods to work with multiple selections, it requires to add selections one by one(what find-and-replace does) & as far as merge performs on every addition, that cause O(N^3) complexity to select-all, where N - count of selections.

So now it takes about 30sec. to select all new-lines in file, that contain 1000 empty lines. Is that "good enough"? In SBT3 it takes less than a second for instance. screenshot from 2016-03-25 17 03 05

nathansobo commented 8 years ago

It's definitely not good enough, though opening a specific issue addressing select-all-occurrences and posting your saved timeline data would be helpful. We have some challenges due to the synchronous nature of Atom's editor APIs. We need to process selections in a batched manner.

matthew-dean commented 7 years ago

This is still a severe issue on version 1.12.7 of Atom and 0.202.2 of find-and-replace. I would replace "not good enough" with "probably the worst large file loading and searching of any JS-based editor". https://github.com/ajaxorg/ace handles large files, loading and searching, very smoothly.

jpike88 commented 7 years ago

Pretty awful. Switched from Sublime Text to this and it freezes up on a medium-large javascript file.

caleyshemc commented 7 years ago

+1, having this issue doing search and replace on large HTML files.

kingjr commented 7 years ago

+1, having same issue on small CSV file

matteopiccioni commented 7 years ago

This bug has been present for too long. I thought I had finally found my osx editor but this is a critical issue for me. While replacing large files Atom version 1.2.3 freezes I am sorry fo this Thanks for your work

nathansobo commented 7 years ago

We now have the infrastructure in place to populate the index in a background thread at the native layer, but we haven't utilized it yet. So there's progress, but we're not there yet.

gradrix commented 7 years ago

What is the progress of this issue? How may I help? You could share the code you are working on, links to specific code files?

This issue is the only thing preventing me to call Atom the best editor! :)

rpbaltazar commented 6 years ago

I had opened an issue in atom's main repo but apparently should have done it in this, or follow up with this thread.

Prerequisites

Description

This is a particular weird issue in the sense that I can't reproduce it across machines. I'm inclined to say that it has to do with machine hardware as it is the main difference between the two testing machines.

As you can see on the attached gif, whenever I search within a project, the process of searching simply hangs and increases the memory consumption up to the roof. I've tried everything I can remember of:

This seems to be project dependent, thus my idea that it might be due to poor performance being beaten by a better machine specs. Are there any issues when the number of files in a project is large?

Steps to Reproduce

atom-memory-bug

  1. Open Atom with a project
  2. Search in project

Expected behavior: Results to start appearing on the output window

Actual behavior: The process hangs and the memory bloats

Reproduces how often: Every time in this machine

Versions

$ atom --version
Atom    : 1.23.3
Electron: 1.6.15
Chrome  : 56.0.2924.87
Node    : 7.4.0

$ apm --version
apm  1.18.12
npm  3.10.10
node 6.9.5 x64
atom 1.23.3
python 2.7.10
git 2.14.3

macOS High Sierra - v. 10.13.3 iMac Late 2013 3.5GHz Intel Core i7 16 GB 1600 MHz DD3

LukeOeding commented 6 years ago

I'm having this issue with an 82kb file which contains 9065 instances of "[" that I want to replace with "_". It's strange because apple textedit does the replacement in an instant. I'm not familiar with this code, but perhaps having search-replace functionality executed via something like sed on the backend would help? Maybe this already is the case?

jedwards1211 commented 6 years ago

Similar to this issue, for every replacement it makes it splices the replaced marker out of an array, which is shamelessly wasteful. I was pretty shocked to discover that when reading the code.

jedwards1211 commented 6 years ago

@LukeOeding it's merely because the way the replacement is implemented in this package is shamelessly inefficient. It's really shaken my confidence in the authors of this package. Other editors are obviously doing it properly because there is no good reason replacing 9065 instances of "[" should take any more than an instant.

jedwards1211 commented 6 years ago

@jpike88 On the other hand, Sublime Text's go to anything has sometimes had abysmal performance for me...we can't win 🤦‍♂️

jedwards1211 commented 6 years ago

@lllShamanlll Do you have a link to code that has the O(n^3) runtime you mentioned? Or has the offending code been improved in the past two years? I identified an O(n^2) performance problem in replace (and made a PR to fix it) but as far as finding goes, createMarkers mostly just delegates to text-buffer/superstring and so far I haven't found any red flags in those packages.

MarkerIndex appears to be a tree, so I assume superstring's find_and_mark_all_in_range is O(n log n) (n inserts * log n per insert) which is not too bad, though unfortunately not as good as doing a replace from scratch without building up an index of markers, which would be O(n).

lee-dohm commented 6 years ago

@jedwards1211 your comment was deleted as a violation of the Atom Code of Conduct as it was insulting or derogatory. You may consider this an official warning.

jedwards1211 commented 6 years ago

Okay, I'll reproduce the non-derogatory part in case @nathansobo didn't see it... If the plan is merely to move the current algorithm to a background thread, that won't make it go any faster, it will just prevent Atom from freezing while the work crawls along on the background thread. So I think the first priority should be to make the algorithm efficient enough for good performance on all but extremely large files, and then worry about moving work to a background thread.

MeerKatDev commented 5 years ago

I wanted to find and replace with regex some chars on a million-chars file. It just freezes, it doesn't even try, it looks like. Ubuntu just asks whether it should close or keep waiting. I'm struggling to believe that after three years this issue wasn't solved.

matthew-dean commented 5 years ago

@MeerKatDev Use VSCode. Better architecture, all the same features and a few more, and doesn't have these issues.

MeerKatDev commented 5 years ago

@MeerKatDev Use VSCode. Better architecture, all the same features and a few more, and doesn't have these issues.

Thanks, but I've used Notepad++ on Windows the last 6 years, everything's good and well. I wanted now to try some substitute for that editor, and Atom looked promising. TBH, Notepad++'s Find and Replace is the fastest ever. This job I'm describing would have been done in a matter of 2-3 seconds.

jaypinho commented 4 years ago

This is still a major issue on Atom 1.47.0 (running on a 2017 MacBook Pro 3.5GHz Dual-Core Intel Core i7 with 16GB of RAM, on macOS Catalina, v10.15.5). I attempted the following find/replace regex on a list of 84,331 URLs (to convert them to domains):

Find: ^http[s]*://([\w\d\.-]*)/.*$ Replace: $1

In Atom I gave up after over 4 minutes of the app not responding (except to ask me every 30 seconds if I wanted to continue waiting for the editor to respond). In Sublime Text v3.2.2, the entire operation took 2 seconds.

basvijfwinkel-cinarra commented 3 years ago

This is still a major issue on Atom 1.56.0 x64 (running on a 2017 Macbook Pro with 16GB of RAM on macOS Catalina).

To test this issue, just create a text file with 80,000 lines and just add 8 random 1s and 0s on each line. Now try to replace the zeroes with some other characters (no Regex neede here) I gave up after 15 minutes and clicking away the OS notification that Atom is not responding. On Notepad++ this takes a few seconds to complete.

theblueskies commented 3 years ago

FYI this issue is back. MacOS 11.4 and Atom version 1.58.0 (running on 2018 Macbook Pro with 32GB RAM)

batmanbury commented 3 years ago

Is this the same issue as find-in-project just being incredibly slow? I downloaded the latest Atom, on macOS Big Sur 11.6, and cmd+shift+F takes anywhere from 15 to 30 seconds, whereas Sublime takes about a second.

These are just basic full project searches, no regex, etc.

mrdc commented 3 years ago

I can confirm: search is extremely slow - I've tried 50MB JSON with 480k lines.

Atom 1.58.0
macOS Catalina 10.15.7
RAM 16GB
i5 3,6Ghz

Related to this issue: https://github.com/atom/find-and-replace/issues/620

khal3d commented 2 years ago

I have a MacBook Pro 2021 M1 Max with 32GB of RAM, and it's really impossible to find and replace "\n" with ", " for a 25k lines file. I had to download Sublime editor to get the job done :(