Editor Code-Completion of Interpreter Namespace Objects (Enhancement)

[GoogleCodeExporter]

Describe the Enhancement:

We would like to have the ability to use Code Completion in the Editor for 
objects that have been created in the Interactive Interpretter. 

This could be an option the user sets, and probably if the "Reinitialize before 
run" option is disabled then we enable this new functionality.

For Example:

In the Interactive Interpreter create a variable
>>> import numpy
>>> data = numpy.zeros((10,10))

Then we would like the Code Completion in the Editor to help complete the 
following:
data.<CODE COMPELTE HERE>

Currently IDE's like IDLE and PythonWin have this feature always on.

Original issue reported on code.google.com by Chris.W....@gmail.com on 5 Aug 2011 at 2:47

[GoogleCodeExporter]

Implemented in Version Control.

You need to add to your editor file:

from __main__ import *

Original comment by pyscripter on 11 Mar 2012 at 12:23

• Changed state: Fixed
• Added labels: Type-Enhancement

[GoogleCodeExporter]

Thanks for this improvement! However, I'm testing it in 2.5.0.0 x86 and I've 
run into two problems:

1. Too Slow - With large libraries loaded into the interactive interpreter, the 
code completion in the Code Editor take almost 10 seconds to create and display 
the  list. Obviously, this makes the feature unusable.

The Code Complete for this same library is very fast in the Interactive 
Interpreter, so I assume there is some bug causing this problem.

To replicate, try importing numpy into the interpreter and trying code 
completion in the editor. (This is still faster than our library as this take 
about 4 seconds compared to ours which is almost double that).

2. Code Completion does not work for lists of objects:
Trivial Example code to replicate the issue:

Add the following to the interactive window:

class Knob():
    def Turn():
        pass

class Radio():
    def __init__(self):
        self.Knobs = [Knob(), Knob()]

CarStereo = Radio()

In the interpreter the following Code Completion works fine:
CarStereo.Knobs[0]. <= Works!

Doing the same in the Code Editor (with the from __main__ import *) does not 
work.

Original comment by Chris.W....@gmail.com on 19 Mar 2012 at 3:19

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

It seems that my item #1 above only happens using the Remote (Wx) python 
engine. Using "Internal" this slow down doesn't appear to happen.

Original comment by Chris.W....@gmail.com on 19 Mar 2012 at 3:56

[GoogleCodeExporter]

I am not sure why.  Here (tested with python 3.2.2 64bit) numpy code completion 
comes up instantaneously in the editor (Core I5 laptop).  Any clues so I can 
address the issue?

As to your second point you need to understand that code completion in the 
editor works differently and will never be as good as in the interpreter.  Code 
completion in the editor never executes code or evaluates expressions.  It is 
just using introspection on live objects. So you have to live with such 
limitations.

Original comment by pyscripter on 19 Mar 2012 at 4:47

[GoogleCodeExporter]

1) I'm running a 3 year old laptop, so this is probably part of the issue.
Seemingly the main clue is that the Internal python engine is much faster than 
the Remote (Wx). 

The fact that the code completion in the Interactive Interpreter runs smoothy 
makes me think it's a code issue and not a performance issue. I don't know if 
there is any background caching going on in the interpreter, but not in the 
editor...

I'm currently running: 
Python 2.7.1 (r271:86832, Nov 27 2010, 18:30:46) [MSC v.1500 32 bit (Intel)] on 
win32. (On XP SP3)

Perhaps that shows a different behavior on your system than 3.2.2?

2) Understood. Fair enough. 

Thanks!

Original comment by Chris.W....@gmail.com on 19 Mar 2012 at 5:13

[GoogleCodeExporter]

Just to make sure.  
a) You start PyScripter with the remote engine.
b) In the interpreter you issue
>>> import numpy
c) In an editor window you type
from __main__ import *
numpy.

and you wait 4 seconds?  This cannot be explained by the speed of the computer. 
 Something else must be affecting performance. As I said here it runs 
immediately. 

PyScripter communicates with the remote python engine with TCP/IP.  

Is some firewall slowing things down?  
Can you try on a different machine to see whether the issue exists?

Explanation
Please note that the speed of code completion and the editor will not be the 
same.  When you type numpy in the interpreter code completion evaluates the 
expression "numpy" and gets the namespace of the result.  On the other hand in 
the editor code completion examines the whole module __main__ locates numpy in 
its namespace and then introspects numpy.  So the heavier __main__ is the 
slower this feature will be.

Original comment by pyscripter on 19 Mar 2012 at 5:32

[GoogleCodeExporter]

Your description of my behavior is accurate.

I'm running PyScripter on my Work laptop right, perhaps you're right and 
Symantec Endpoint Protection is to blame. This evening I'll test again on my 
home PC.

(Thanks for your prompt feedback)

Original comment by Chris.W....@gmail.com on 19 Mar 2012 at 5:42

[GoogleCodeExporter]

Okay, I tried again on my home PC which is running Windows 7, a much newer and 
faster machine. It has the same issue, in fact I would estimate the delay 
between typing the period and the code completion window opening is longer than 
my slower work laptop.

I spent some time trying to ensure all the Windows firewalls are wide open, and 
I think all the settings are okay.

I have another personal laptop running XP SP3, same version of Pyscripter and 
Python, yet this one appears to work correctly. This laptop is much slower, but 
the code complete time in the editor matches closely with that in the 
interpreter.

(I should also note this laptop is also not seeing the KeyboardInterrupt 
problem I mentioned here: 
http://groups.google.com/group/pyscripter/browse_thread/thread/6199fac994b9417c 
)

This looks to be a tricky problem to debug. Anyone else having similar problems?

Original comment by Chris.W....@gmail.com on 20 Mar 2012 at 1:51

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

It (the steps in comment 7) takes about 2 seconds on my (very slow) laptop when 
I'm using the *remote engine*. When I type `numpy.` in the interpreter, on the 
other hand, it takes <500 ms. Using the *internal engine*, autocomplete happens 
quickly in both the interpreter and the editor.

I'm running 
Python 2.7.2 -- EPD 7.1-2 (32-bit)
Pyscripter 2.5.1.0 x86 (would be nice if I could copy this from the help->about 
screen, by the way)

Thanks for implementing this feature! To make it more discoverable, I would 
consider adding a checkbox to the context menu for the interpreter that says 
"Add interpreter code completion to editor" and make that a global option. Or 
something like that :-)

Original comment by pckuj...@gmail.com on 20 Mar 2012 at 5:05

[GoogleCodeExporter]

Could you please test with version 2.5.3.  I have made some changes that may 
have improved the speed of completion of live objects in the editor.

Original comment by pyscripter on 24 Mar 2012 at 3:20

[GoogleCodeExporter]

I upgraded and now PyScripter won't open. I also tried uninstalling and 
reinstalling and it throws the following error:

Python could not be properly initialized. We must quit.

If I use the open latest version shortcut, the 2.7 or 2.6 shortcut it also 
throws this error:

Error 193: Could not open Dll "python26.dll"

I'm guessing it's a new bug in PyScripter, but perhaps something in my Python 
install got messed up... let me know what you think.

Original comment by Chris.W....@gmail.com on 24 Mar 2012 at 6:39

[GoogleCodeExporter]

By accident, I included the 64bit executable in the 32-bit distribution.  
Sorry.  Will reupload tonight.

Original comment by pyscripter on 24 Mar 2012 at 10:00

[GoogleCodeExporter]

No problem. I'll have a look once it's up. Thanks!

Original comment by Chris.W....@gmail.com on 24 Mar 2012 at 4:31

[GoogleCodeExporter]

Okay, PyScripter runs again.

It seems the live code completion is a little bit faster, but still slow enough 
that it's not usable for me. The fact that my 10 year old home laptop works 
fine, while my new PC doesn't makes me think something else besides PyScripter 
is causing the slow down. When I have more time I'll do a full reinstall of 
Python and all the libraries.

Original comment by Chris.W....@gmail.com on 24 Mar 2012 at 11:25

[GoogleCodeExporter]

Chris,

The mechanism for introspection of live objects in the editor is now the same 
as the one used in the variables window.  Do you see the same issue in the 
variables window? i.e.  if you do 
>> import numpy 
in the interpreter and then go to the Variables window and try to expand numpy 
do you see similar delays?  (Here it is instant).

You say the old machine is faster than the new one.  How many cores does your 
new machine have?  I wonder whether this is due to some contention due to 
Python's Global interepreter lock.  However I have seen no such issues on my 
dual core computers.

I have seen some other reports about slowness (e.g. Issue 602) but it appears 
to be a rare exception rather than the rule, since PyScripter is used widely 
and often for serious business Python development.  I would be very interested 
though to find out what are the causes of this problem.

Original comment by pyscripter on 25 Mar 2012 at 12:42

[GoogleCodeExporter]

So, expanding the numpy in the variables window as you described is instant, 
while the editor is slow. Interesting.

My new machine is an AMD Athlon II X4 635 Processor 2.9GHz (So, 4 cores). 
Running windows 7 - 64 bit.

My Laptop is a single core AMD Turion 64 Mobile running XP SP3.

My work laptop also experiences this issue, unfortunately I can't check its 
specs till Monday. It runs XP, but I can't remember off-hand if it's single or 
multi core, or if it's Intel or AMD.

I wonder if it has something to do with Multi-core AMDs... Or maybe I should 
try using the 64-bit Python?

Original comment by Chris.W....@gmail.com on 25 Mar 2012 at 1:47