Komodo Edit 10.1.4 becomes laggy after a while

[garybentley]

Short Summary

After being open for a while and having about 20 files open Komodo Edit becomes sluggish and inputs take a second or so to process. For example, opening the goto line will often miss the first couple of numbers I enter in the text box. Similarly the "Go to Anything" bar will take a couple of seconds to find the correct file/directory and if I don't pause for a little while I will often open the wrong thing.

These problems don't happen when I first open Komodo Edit. All the buffers open are .java files. No addons have been installed/enabled. The problem happens with .js files as well. The problem remains even if I close all the files, I have to restart KE to clear the problem.

Steps to Reproduce

Open 20-30 files, .java and .js seem to trigger the problem for me.

Expected results

Not be laggy?

Actual results

Is laggy after about 30-40 minutes of editing the files.

Platform Information

Komodo Edit, version 10.1.4, build 17456, platform win32-x86. Built on Fri Dec 16 19:15:05 2016.

Windows 10 64-bit.

Additional Information

There are a number of errors in the log file but I can't attach the file and the file is too big (230KB) to be pasted here.

[Naatan]

Are you saying this started happening with 10.1.4 or is this just the first version you happen to be using?

[garybentley]

It started with 10.1.4 and the previous dot version. I've been using Komodo Edit since version 4.

[Naatan]

Could you verify that version 10.1.2 runs without lagging? (as per your comment that's the last version that ran properly).

http://downloads.activestate.com/Komodo/releases/archive/10.x/10.1.2/

[garybentley]

It won't install 10.1.2 because it detects 10.1.4. If I uninstall 10.1.4 will I lose all my settings for KE? I don't want to have to rebind all the keys.

[Defman21]

You shouldn't lose your settings if you uninstall Komodo, but I recommend to back up your profile folder before doing that.

[Naatan]

You won't loose any settings, like @Defman21 said. No backup necessary.

[garybentley]

I've installed the 10.1.2 version, I'll let you know what happens. As an aside, the uninstall didn't completely remove KE. I was still left with a C:\Program Files (x86)\ActiveState Komodo Edit 10\lib folder that contained "mozilla", "sdk" and "support" folders. I had to remove that folder before installing again.

[garybentley]

I've been running version 10.1.2 for about 2 hours now, with the same sort of usage as I did in 10.1.4, and I'm not seeing the same lag occurring. I'll continue for a few more hours and then do another update.

I notice that version 10.1.2 has a substantially lower memory usage than 10.1.4. It's currently running at around 220MB compared to 450MB (for 10.1.4) for the same number of open files. 10.1.4 also had a base cpu usage of around 5% as if a background process/thread was running, 10.1.2 doesn't have that base usage.

[garybentley]

The slowdown has now occurred in 10.1.2. KE is now consuming 355MB and whenever I type or move the scrollpane there is noticeable lag and the cpu usage jumps to 5/6% for a while, even after I stop typing.

I have 37 java files open. Closing some tabs doesn't help, the memory usage remains constant and when I close a tab the cpu usage jumps to around 10% for about 10s.

[Naatan]

@garybentley very interesting. Please continue using Komodo as you have been but be mindful of when the problem starts occuring. If you can figure out what "triggers" it we may have something here.

[cavac]

I found one bug that may be related to this problem: I'm using mercurial integration, and after a while i get a defunct "hg" process:

$ ps aux | grep hg
cavac    22028  0.0  0.0  14540   936 pts/16   S+   17:08   0:00 grep --color=auto hg
cavac    30123  0.0  0.0      0     0 ?        Zs   12:21   0:00 [hg] <defunct>

My guess is that Komodo doesn't handle the SIGCHLD signal properly. So, next time Komodo wants to access mercurial, it may try to use an already terminated process.

[Naatan]

I highly doubt that's related to Komodo slowing down over time. defunct processes using 0 CPU wouldn't really affect performance.

[BryceLBL]

I'm seeing this issue also. No other plugins but working on python and other files.

[bluepowder]

Same here, with about 10 Smarty Templates, 10 PHP, 5 Javascript, & 5 CSS files open over sftp. Komodo Edit slowly becomes unusable and has to be reopened. This is on a computer with an i7, 16GB RAM, an SSD & a fast internet connection. No plugins, and I even tried reinstalling (after cleaning up the AppData. This is a problem with both the release version and the nightly.

[th3coop]

I don't think this is related directly to number of files open. I'm currently working with 30 different files open and not noticing an issue. There must be some other commonality here. Is everyone using Komodo Edit? I'm using Komodo IDE.

[cavac]

Using: Komodo IDE, version 10.2.2, build 89888, platform linux-x86_64. Built on Thu Apr 13 00:37:21 2017

I've been watching it a bit more closely, but i still have no really reliable way of reproducing the issue quickly. I know it happens after some time, in my case mostly about a day or two when the IDE is mostly idling. When doing a lot of work, constantly opening/closing files and restarting the debugging sessions, it seems to happen more quickly, but i can't be really sure.

The one thing i noticed on my installation is, that it doesn't seem so much a gradual slowdown, but it more or less seem to happen in a short time when it happens.

Next time i run into trouble, i'm gonna try to find out more.

[th3coop]

Thanks for the udpate @cavac. Are any of you consistently running Komodo with multiple windows open?

[cavac]

I have Komodo constantly open (nearly 24/7), but in the usual one-window-many-tabs configuration.

[cavac]

Here's the memory usage from a fresh restart of Komodo. memory-report_after_restart.json.gz

And here it is a few days later with the slowdown problem (had a long weekend in between, which did NOT slow down Komodo. Only after working with it for a couple of hours after the weekend did it happen): memory-report_with_slowdown.json.gz

Komodo logs: py_logs.tar.gz

CC log: cc-edges.29035.1493715277.log.gz

GC log: gc-edges.29035.1493715277.log.gz

[th3coop]

Great. Thanks for the info @cavac.

[th3coop]

Perhaps interesting finding. I've set a script running in the JS console that opens 4 files, lets them sit for about a second, closes them all, then reopens them all again. After running for about 10 minutes Komodo is fluctuating between 300MB of Memory and 3GB. Huh, it actually appears to have become unusable. This may be an avenue of investigation for us though I need to wait until the scripts stops then I can see if things go back to normal or not.

Will report back when script stops.

[th3coop]

BAH, no dice. Once the script finished Komodo became quite snappy again. Memory and CPU usage when down to normal numbers as well, 0-14% and 300-400MB respectively.

[cavac]

@cgchoffman Try changing the script so it modifies the files, saves them, THEN closes them. Might have something to do with the internal diff stuff (changebar in the edit window) or with the save function itself.

At least, this should more closely resemble a normal workflow.

[th3coop]

@cavac, good idea. No effect unfortunately. The script opens 5 files, edits each one, sets the focus on it so various tools run on the file, saves, closes and repeats for 30 minutes. Komodo's running smooth after that. For now I think we can eliminate opening, edit, closing files.

Any other commonalities in workflow? A lot of version control commits? Lots of debug sessions?

[th3coop]

If anyone is interested they can play with the Userscript too: open_close_files.txt Just rename it to open_close_files.komodotool and drop it on Komodo. You'll have add your own path and lists of files.

The timing is a little wonky in there. Not the most realistic workflow.

[cavac]

Let's see if anyone can match my major points:

• Mercurial SCM with lots of changesets (only to display, actual SCM stuff is done in command line)
• Perl
• some perl scripts use forking (Net::Server prefork)
• Perl scripts modify @INC during startup
• HTML/Template Toolkit
• XML files
• frequent debugging sessions
• Projects with quite a few "Add existing folder" folders (i think they where called "live folders" at some point)
• thousands of lines in the Debug Output pane
• Minimap is active
• lots of adding/removing breakpoints
• Linux (Ubuntu 16.04 64 Bit)

Update: Progress pretty much slowed down to non-extistant in Loop 6. It now takes 4-5 minutes(!) to open some files.

[cavac]

I have modified the script a bit to work in stages for a more realistic simulation of work. openfiles.komodotool.txt

I am watching it run as we speak. it is now in the 3rd loop, and it takes up to 15 seconds to open a file. Closing files also sometimes hangs for up to 30 seconds or so.

The project files i am using are these: https://cavac.at/public/mercurial/maplat_framework/

A hg clone https://cavac.at/public/mercurial/maplat_framework/ and editing the komodo script to point to the correct path should do the trick.

Not sure yet if Komodo will settle down and get responsive again after the script finishes. But opening or saving a text file should never take 15+ seconds (the biggest file in there is around 30kB). As for needing 30 seconds to close a file, that's just mind boggling slow.

Also, while Komodo is opening a file, the user interface is completely non responsive. Can't even scroll in the console or anything. I'm running with mercurial integration enabled.

Is there a way in userscripts to log to a file or export the contents of the console?

[Naatan]

@cavac does running this script in a fresh profile (Help > Troubleshooting > Restart in safe-mode) reproduce the issue?

[cavac]

Hmm, i installed the latest nightly and now i can't reproduce the results of my script. Might be coincidence, though. I'll keep track of the issue.

[pixxelfactory]

I'm working with a lot of large php and scss files (today scss mostly), and after a couple of hours it becomes reaaaaally laggy, autocompletion is pretty useless after a while, since it takes so long to load them when i'm typing, that i finish writing the word(s) first. After a simple restart of komodo (edit), its fast again (with the same files open). The system itself is not the problem, recently switched from an old imac to a custom-built win10 pc with a ryzen 2700x, 32gigs of ram and 970evo as main disk and 860evo as data-storage.... =/

[th3coop]

@pixxelfactory , could you give the latest nightly a shot? Quite a few performance enhancements have been submitted to the repo, a few specifically for PHP, so it might be worth trying.

http://downloads.activestate.com/Komodo/nightly/

[pixxelfactory]

@th3coop , thanks, i'll try it out!

[th3coop]

@pixxelfactory, let us know how it goes.