Editor has become virtually unusuable because of code completion

[spyder-bot]

From steve.f....@gmail.com on 2011-05-19T11:26:38Z

My editor is constantly locking up rendering spyder nearly unusable. Unsure exactly what's going on. CPU usage is not spiking when the editor locks up.

What steps will reproduce the problem?

1. Edit files

I already have pyflakes disabled and performance analysis set to 10000 milliseconds. I am changing the performance analysis to only when saving file to see if that will help at all. What version of the product are you using? On what operating system? Spyder 2.0.11, Windows XP SP3, python 2.6.1

Original issue: http://code.google.com/p/spyderlib/issues/detail?id=659

[spyder-bot]

From ccordoba12 on 2011-05-27T08:29:31Z

It runs fine for me in my virtual machine with Python(x,y) 2.6.6 and I've never seen a problem like this on Linux.

Maybe it has to do with a problem updating to a newer version of PyQt4, like the problems reported on issue #636 ?

Labels: OpSys-Windows Cat-Editor

[spyder-bot]

From steve.f....@gmail.com on 2011-05-27T09:23:23Z

Yeah, I have no idea what's going on. I'm seeing consistent performance problems with spyder. The editor becomes locked up and unresponsive for seconds at a time. Sometimes text I type in the editor doesn't show up until I hit enter or a few seconds later. Maybe I should try resetting to stock settings and see if that changes anything?

[spyder-bot]

From steve.f....@gmail.com on 2011-06-08T14:09:27Z

Okay. So I've figured out a few things. The slow downs I'm experiencing are caused by auto completion features. It's always when I'm typing '(' or '.'. Usually the editor sit there doing almost nothing and then after a few seconds I'll see the rope pop up or the member box pop up.

I turned off the code completion option for '.' items which caused those associated slow down to go away. The slow down now only occurs when I hit ctrl+space to attempt to auto complete something. Once I disable balloon tips, '(' no longer causes slow downs.

So, it's definitely the auto-completion routines. Spyder is now fast, but without auto completion functionality Spyder has lost one of its most beneficial features. This is a critical issue for me. I'm not really sure what changed over time because this was not always an issue. My functions tend to have very large docstrings because they are doxygen compliant. I'm not sure if this is a factor or not. I can't really go through my entire code base.

Additionally, I'd like to see all the items under the code introspection section in the options get tool tips. It looks like only the last two items currently have tool tips.

[spyder-bot]

From steve.f....@gmail.com on 2011-06-08T14:26:30Z

Also, Ctrl+Clicking is slow. I'm wagering it uses the same lookup mechanism as the auto-completion stuff.

[spyder-bot]

From steve.f....@gmail.com on 2011-06-14T15:56:30Z

Another point of interest: When I open spyder it uses ~90 MB of RAM. Over time I've seen its RAM usage balloon to over 400 MB!

[spyder-bot]

From pierre.raybaut on 2011-06-15T00:41:54Z

The performance of rope-powered features (code completion, go-to-definition, calltips) are related to the 'rope' version which is installed on your machine.

If this is rope v0.9.2 or v0.9.3, it should be patched by Spyder to improve performances: https://code.google.com/p/spyder/source/browse/spyderlib/rope_patch.py#16 See this post: http://groups.google.com/group/spyderlib/browse_thread/thread/b8f9d4c4565cf199/d9eaa24b3c1d1e00

[spyder-bot]

From steve.f....@gmail.com on 2011-06-16T07:29:12Z

I have looked in my site-packages directory and I do have rope 0.9.3 installed. How can I tell if rope has been properly patched?

[spyder-bot]

From steve.f....@gmail.com on 2011-06-16T07:37:08Z

I ran the rope test and got the following output:

D:\Devel\spyderlib\rope_profiling>test.py completion: 30 ms False Testing other rope instrospection features

elapsed time: 20 ms

calltip: None definition location: (None, None) doc: * DOCUMENTATION **** * None

---

*

[spyder-bot]

From steve.f....@gmail.com on 2011-07-12T08:07:56Z

Has anyone ever had a change to try to take a look at this? My colleagues and I are forced to turn off some of the best features of Spyder in order the make the editor usable. I'd really like some assistance looking into this issue.

[spyder-bot]

From steve.f....@gmail.com on 2011-07-12T10:02:22Z

Actually, I went ahead and re-enabled things to see how they are performing. Performance seems to be a little better, so I'm going to keep testing and see if issues arise again.

[spyder-bot]

From pierre.raybaut on 2011-07-12T13:32:10Z

Ok, thanks for the feedback and keep up posted on whatever your conclusions are.

Anyway rope performance issues seem to be cyclic: the patch was supposed to fix this but the other day I found myself wondering why my powerful quad-Xeon machine was taking so much time to popup calltips for Python built-ins like 'sorted'. I didn't have the time to check it out but the fact that it was not instantaneous is quite surprising and made me suspicious regarding the current patch.

[spyder-bot]

From steve.f....@gmail.com on 2011-08-30T16:08:04Z

Unfortunately my conclusion is that the majority of the time code completion/call tips performance is very poor for me. There are occasions where it works fluidly, but most of the time the performance is no good. Have you considered putting in some kind of monitoring code to watch for performance aberrations? If I had more time I would do it. I would definitely provide feedback on the reported data if you were to do so.

[spyder-bot]

From pierre.raybaut on 2011-08-31T01:46:21Z

There is already a debug mode in spyderlib/widgets/sourcecode/codeeditor.py: simply replace DEBUG = False by DEBUG = True.

There is also a rope profiling folder in source package.

The best would to compare performances with another editor using rope to provide these features: there is Vim for example.

[spyder-bot]

From steve.f....@gmail.com on 2011-09-02T10:25:53Z

2.0.12's codeeditor doesn't seem to have that variable defined. Is this something in the 2.1 beta?

[spyder-bot]

From pierre.raybaut on 2011-09-02T10:37:33Z

Absolutely.

[spyder-bot]

From steve.f....@gmail.com on 2011-12-02T14:18:57Z

I've enabled that flag but not really sure where I should be looking for useful output. In general, I'm still seeing pretty poor performance when doing code completion lookups.

[spyder-bot]

From techtonik@gmail.com on 2011-12-02T15:01:46Z

If somebody could figure out an entrypoint for autocompletion magic, it may be possible to check if lookup can be sped up with balanced trees instead of lists (I doubt that rope authors didn't thought about this, but still..).

[spyder-bot]

From techtonik@gmail.com on 2011-12-02T15:03:34Z

Another thing that may help is http://docs.python.org/library/bisect.html

[spyder-bot]

From ccordoba12 on 2011-12-02T16:44:33Z

Steve, after enabling DEBUG you have to look for the file rope.log in your .spyder2 folder.

I've been working on a similar bug the last couple of weeks (see issue #853 ). What modules are you seeing the autocompletion problems with? Are your own libraries or third party ones?

As you correctly guessed, these problems are related to rope and I think all of them can be solved by adding the necessary modules to ROPE_PREFS['extension_modules'] in spyderlib/widgets/sourcecode/codeeditor.py

By now I'm working on how to correctly generate calltips and extract docstrings after adding modules there.

Summary: Editor has become virtually unusuable because of code completion
Labels: -Cat-Editor Cat-CodeCompletion

[spyder-bot]

From steve.f....@gmail.com on 2011-12-02T18:54:32Z

I've seen the slow behavior on both built in python stuff and third party modules.

[spyder-bot]

From techtonik@gmail.com on 2011-12-03T00:24:56Z

One more idea for improvement is using probability theory - http://norvig.com/spell-correct.html

[spyder-bot]

From steve.f....@gmail.com on 2011-12-06T13:05:52Z

I just thought I'd paste in some info from my rope.log:

Logging time: Tue Nov 29 11:36:22 2011

get_definition_location: 8700 ms

Yeah. 8.7 seconds. Fail worthy right there.

Logging time: Tue Nov 29 11:38:12 2011

code_assist/sorted_proposals: 940 ms

I've got a couple of things from the debug log as well that are creating a lot of noise. Shall I paste them here?

[spyder-bot]

From steve.f....@gmail.com on 2011-12-06T13:29:00Z

Logging time: Tue Dec 06 15:27:22 2011

get_definition_location: 10300 ms

IN THE SAME FILE FFS!

P.S. It might be nice if the DEBUG variable could also be set off an environment var so I don't have to edit my spyder codeeditor file.

[spyder-bot]

From pierre.raybaut on 2011-12-06T14:13:41Z

Regarding the environment variable toggling the debug mode, I think that's a very good idea. Maybe we could use it to turn on the debug modes of all modules.

The stats you are reporting indeed reflect very poor performance. You definitely proved your point but what we would really need to know is the exact situations where these catastrophic results were obtained. We need to be able to work with the same code sample which reproduces systematically this behavior. That's what I tried to do with the "rope profiling" folder in Spyder's source package: I had performance issues too and I came up with this test to reproduce those bugs and finally solved them. Obviously this is not working for you (and most probably for others), so we need to be able to reproduce this in order to fix it now and to be able to check for regressions in the future.

[spyder-bot]

From steve.f....@gmail.com on 2011-12-07T07:47:35Z

Well, I can't release our source. Do you have any ideas about how we should go about debugging this?

[spyder-bot]

From ccordoba12 on 2011-12-07T09:29:03Z

Steve, you can do two things:

1. Send us an script in which you're only using standard library modules and seeing the problem
2. Add your own module names to ROPE_PREFS['extension_modules'] in spyderlib/widgets/sourcecode/codeeditor.py to see if you got better completion times or not

Labels: -OpSys-Windows

[spyder-bot]

From ccordoba12 on 2013-02-13T09:54:38Z

Status: Duplicate
Mergedinto: 1272