Code completion in the Editor doesn't work properly, unlike in the IPython console

[spyder-bot]

From marushk...@gmail.com on 2015-02-06T08:44:55Z

Spyder Version: 2.4.0dev Python Version: 3.4.1 Qt Version : 4.8.6, PyQt4 (API v2) 4.10.4 on Windows pyflakes >=0.6.0: 0.7.3 (OK) pep8 >=0.6 : 1.4.6 (OK) IPython >=1.0 : 2.3.1 (OK) zmq >=2.1.11 : 14.4.1 (OK) pygments >=1.6 : 2.0.1 (OK) pandas >=0.13.1 : 0.14.1 (OK) sphinx >=0.6.6 : 1.2.3 (OK) jedi >=0.8.1 : 0.8.1-final0 (OK) rope >=0.9.2 : 0.9.4-1 (OK) matplotlib >=1.0: 1.4.0 (OK) sympy >=0.7.0 : 0.7.5 (OK) pylint >=0.25 : 1.4.1 (OK)

What steps will reproduce the problem?

1. Import module
2. Create a local variable using class from the module
3. Write the name of the variable, put the dot and call the autocompletion (TAB or CTRL+SPACE)

What is the expected output? What do you see instead?

In the editor: I expected popup box with available methods and members, but it don't appear. In the console: All is OK.

Please provide any additional information below

I tried numpy and cv2:

import numpy as np # 'imp' completion works, 'num' completion works
arr = np.array([1,2,3,4,5]) # 'np.' completion works
arr. # completion don't work 

import cv2
cap = cv2.VideoCapture(0) # 'cv2.' completion works
cap. # completion don't work 

Same code in console work fine. I use Anaconda and last spyder version from bitbucket and run it like follow: "python bootstrap.py --debug" When I call completion for variable arr. or cap. I got next debug output: completions request: [] completions request from jedi complete: "[]" in 0.1 sec completions request from rope complete: "[]" in 0.2 sec got timeout

Original issue: http://code.google.com/p/spyderlib/issues/detail?id=2162

[spyder-bot]

From ccordoba12 on 2015-02-08T09:36:09Z

Steven, do you think we can do something about this? I see two options here:

1. See if Jedi supports something like type hinting that can help us in this case.
2. I did a bit of research on the Canopy's architecture and noticed its Editor seems to do code completions using an IPython kernel in the background. Of course, the problem in this case is that code in the Editor can be syntactically incorrect, but given we can verify that by using pyflakes, it could be worth investigating this option.

Status: Accepted
Cc: steven.s...@gmail.com
Labels: Cat-Editor MS-v2.4

[spyder-bot]

From ccordoba12 on 2015-02-08T09:43:05Z

issue #1816 has been merged into this issue.

[spyder-bot]

From steven.s...@gmail.com on 2015-02-08T10:20:18Z

This is a really tough one, because it is a compiled module, the best I can suggest is to try docstring typehinting: http://jedi.jedidjah.ch/en/latest/docs/features.html#type-hinting .

[spyder-bot]

From ccordoba12 on 2015-02-08T11:14:02Z

I see, but that wouldn't help for the code reported by the OP, would it?

And what do you think about option 2? Is it feasible?

[spyder-bot]

From steven.s...@gmail.com on 2015-02-08T11:19:39Z

Ah, I missed the numpy.array part. There is an open issue for that: https://github.com/davidhalter/jedi/issues/372 .

As far as IPython goes, I don't think it makes sense, because you don't want your code executing behind your back. What if it is creating/deleting files or doing long-running operations?

[spyder-bot]

From ccordoba12 on 2015-02-08T11:52:28Z

Oh yeah! I missed that one, a crucial point.

Another idea then: does Jedi support a hooks concept for completions? I mean, if jedi knows an object's type (for example, that arr's is a numpy.array), we could add a hook to show completions for it, either by passing the completion widget a list (saved somewhere in our source code) of np.array's completions, or by evaluating this simple statement in an IPython kernel

 import numpy as np
 arr = np.array

then getting the list of completions the kernel returns for foo, and passing it to our widget.

Of course, the crucial point here is that Jedi could give us the type of an introspected object.

[spyder-bot]

From ccordoba12 on 2015-02-08T12:12:12Z

Well, according to this SO answer http://stackoverflow.com/a/14710037/438386 Jedi does give completions for np.arrays, but it seems to have problems in virtualenvs. Steven, can you test it in your side?

[spyder-bot]

From steven.s...@gmail.com on 2015-02-08T18:47:38Z

I tried the following from IPython (Anaconda Python 3.4) and got an empty list:

import jedi; jedi.Script('import numpy; a = numpy.array().m').completions()

[spyder-bot]

From steven.s...@gmail.com on 2015-02-09T08:10:11Z

Carlos, we could back-track to the object definition, then load that in IPython, but there is a snag: np.array is a function, not a class. It also requires an object as its first parameter. We could special case for np.array since it it used so often...

d = jedi.Script('import numpy; a = numpy.array').definition()[0]
if d.name == 'array' and d.module_name == 'numpy.core.multiarray':
   completions = [c for c in dir(np.array(1)) if c.startswith(completion_text)]
elif d.type == 'class':
   mod = __import__(d.module_name)
   cls = getattr(mod, d.name)
   completions = [c for c in dir(cls) if c.startswith(completion_text)]

[spyder-bot]

From ccordoba12 on 2015-02-12T08:43:41Z

Yes, I was thinking exactly that. Could we make this mechanism extendable? I mean, we don't just need np.array, but also pyplot.figure, pd.groupby, and who knows how many others :-)

[spyder-bot]

From steven.s...@gmail.com on 2015-02-12T16:40:53Z

Yes, I suppose we could keep a list of (name, module_name, args) tuples.

[spyder-bot]

From ccordoba12 on 2015-02-13T12:15:15Z

Great!! I promise (really promise) to move to GH this weekend so you can start to work on this :-)

[ccordoba12]

@blink1073, have you had time to look at this issue? People continues to ask for it. See for example:

http://stackoverflow.com/questions/28513322/biopython-intellisense-iin-spyder

[blink1073]

No, not as of yet...

[bilderbuchi]

+1 on this!

[goanpeca]

@blink1073 can you give me some pointers to the completion code, so I can take a look at this ?

[blink1073]

This code would go here, where you'd implement this code, using a constant list of (name, module_name, args) defined in the module.

[goanpeca]

:+1:

[farooqhasny]

hi I am using Spyder 2.3.8 (anaconda) and have similar problem code completion is not working in editor or IPython Console .But it is working in Python console . to regenerate the problem use following code

from lxml import etree
R1 = etree.Element("root")
R1.

[JohnRobson]

"Spyder 2.3.9"

import pandas as pd
df = pd.read_csv("file.csv")
df.(NO code completion)

Completions on assignations is the most desired feature.

Some update on this issue? Tks

[ccordoba12]

Completions for DataFrames will work in Spyder 3.0. For other types we will need to implement the solution proposed here, which is not so easy ;-)

[JohnRobson]

Thank you Cordoba.

I found an interesting workaround for DataFrame Column Names.

df = pd.read_csv("credit-training.csv") # read dataset
>>> df.columns # print columns names

I copy the result and paste it in the bottom of my file as a comment:

... my python code ...

'''
credit-training.csv columns
['serious_dlqin2yrs','revolving_utilization_of_unsecured_lines','age','number_of_time30_59_days_past_due_not_worse','debt_ratio','monthly_income','number_of_open_credit_lines_and_loans','number_of_times90_days_late','number_real_estate_loans_or_lines','number_of_time60_89_days_past_due_not_worse','number_of_dependents']
'''

Now, these "words" are on the editor environment and when I type:

df['m<ctrl+space>']

Spyder will show a window with words starting with 'm' and I choose what I want:

df['monthly_income']

This workaround save me a lot of time ;)

John

[bcolsen]

I didn't write this code but this jedi pull request actually implements parsing the numpydoc doc strings for call return types: davidhalter/jedi#796 https://github.com/Erotemic/jedi/tree/parse_google_docstr

After some effort, I was able to get this: import jedi; jedi.Script('import numpy; a = numpy.ones().m').completions() to actually return something:

Things I did to make this work(Edit: This is unnecessary. See my comment below)

1. I'm using Anaconda on Ubuntu and Python 3.5
2. I had to change the doc string for numpy.ones from out : ndarray to out : numpy.ndarray. adarray isn't a type known to jedi.
3. I was only able to get this to work by importing the jedi code from the source folder. The installed (python setup.py install) versions didn't work. As a result I couldn't get it to work with Spyder. I'm probably doing something wring here.
4. I have to run it 3 times(it's consistent too). According to the Jedi debug function the first two times jedi doesn't even find the numpy.ones function.

My set up: jedi >=0.8.1 : 0.10.0 (OK) matplotlib >=1.0 : 1.5.3 (OK) nbconvert >=4.0 : 4.2.0 (OK) numpy >=1.7 : 1.11.1 (OK) pandas >=0.13.1 : 0.18.1 (OK) pep8 >=0.6 : 1.7.0 (OK) psutil >=0.3 : 4.3.1 (OK) pyflakes >=0.6.0 : 1.3.0 (OK) pygments >=2.0 : 2.1.3 (OK) pylint >=0.25 : 1.6.4 (OK) qtconsole >=4.2.0: 4.2.1 (OK) rope >=0.9.4 : 0.9.4-1 (OK) sphinx >=0.6.6 : 1.4.6 (OK) sympy >=0.7.3 : 1.0 (OK)

I don't know if this is useful but I spent a couple hours on it so I might as well document it here to see if anyone else has better luck. I might try out matplotlib in the future.

[ccordoba12]

@bcolsen, thanks for looking into this!! I think the real solution is what Steve proposed in this comment: https://github.com/spyder-ide/spyder/issues/2162#issuecomment-74779242

I'm going to assign one of our juniors to work on this one for 3.2.

---

@mariacamilaremolinagutierrez, this will be one of your projects for 3.2. Not for you to worry about this one for 3.1 :-)

[bcolsen]

@ccordoba12 You are right. The solution for numpy in general is difficult because of the compiled functions from numpy.multiarray like all of these:

[arange, array, broadcast, can_cast, compare_chararrays,
concatenate, copyto, count_nonzero, dot, dtype, empty,
empty_like, flatiter, frombuffer, fromfile, fromiter, fromstring,
inner, int_asbuffer, lexsort, matmul, may_share_memory,
min_scalar_type, ndarray, nditer, nested_iters, promote_types,
putmask, result_type, set_numeric_ops, shares_memory, vdot, where,
zeros]

Jedi can't find the docstrings for these, because they are not in the function headers.

However, presently I have the Jedi patch working now in Spyder without too much funny business(I didn't have to change numpy or spyder at all):

To install it correctly I had to remove the Jedi that came with Anaconda(which unfortunately removes Spyder) and install the patched Jedi and Spyder manually. Now I don't have to run it three times and I can run it anywhere. np.array doesn't work but my present work around is to use np.asarray instead.

Other good news, Matplotlib also completes (somewhat) with this patch:

I really wanted this...It's hard to remember all the methods for an Axes object

They need to add more numpydoc docstrings to Matplotlib but plt.figure works out of the box. I got subplot and axes to work by writing this at the top of the docstrings for Figure.add_axes and Figure.add_subplot in matplotlib/figure.py

    Returns
    -------
    axes : Axes
       The Axes you made

    Examples
    --------

I'm going to write an issue to Matplotlib and see if the docstrings can be improved. Maybe I can even fix them, though I suspect they will want a better effort than the one I wrote above :-)

[ccordoba12]

@bcolsen, this is very, very interesting progress!! Thanks a lot for looking at it :-)

Would you like to create a PR with the Jedi patches here in Spyder? We already have some patches for rope, so we could do the same for Jedi, and have support for this in Spyder 3.1 (until Jedi maintainer merges the PR you mentioned above and release a new version).

[ccordoba12]

Of course, what we'd need is the part related to Numpy docstrings and not Google ones in davidhalter/jedi#796

[bcolsen]

Patching Jedi. Good idea. I can give that a shot. I should have some time around the end of the month.

I also think I'm close to a hack on Jedi to return types for compiled functions like np.array and scipy stuff, but I'll start with a simple pull first.

[ccordoba12]

Great! Thanks a lot for looking at it :-)

[bcolsen]

Sorry for hijacking this issue more....but I wanted to share my success. I got compiled objects in Jedi to return numpydoc return types. So now np.array() (and most of the other numpy stuff) now completes:

The code is still a little messy but I'm going to attempt to make a usable Jedi branch with the code and maybe another patch for Spyder.

[ccordoba12]

This is really cool!! Thanks a lot for taking the time to look into it.

It'd be great if you could have a patch for Spyder before January 7, which is the feature freeze date for Spyder 3.1 :-)

[hongyan99]

Is this issue fixed? I'm using 3.1.4 and is still not able to do code completion. It does work, however, in the "Python console"

[bcolsen]

@hongyan99 Can you give an example code of what doesn't work compete?

Code completions should work for Numpy in the editor now, and for Matplotlib in the future when they have more complete docs.

The editor has to guess the completions without running your code. This is much more difficult than what the console has to do.

[ceaza]

Still not working for me and running from source: https://github.com/spyder-ide/spyder/commit/efbfd54f1 using winpython 3.6.2.

IPython >=4.0 : 6.1.0 (OK) cython >=0.21 : 0.26 (OK) jedi >=0.9.0 : 0.10.2 (OK) nbconvert >=4.0 : 5.2.1 (OK) numpy >=1.7 : 1.13.1 (OK) pandas >=0.13.1 : 0.20.3 (OK) pycodestyle >=2.3: 2.3.1 (OK) pyflakes >=0.6.0 : 1.6.0 (OK) pygments >=2.0 : 2.2.0 (OK) pylint >=0.25 : 1.7.2 (OK) qtconsole >=4.2.0: 4.3.1 (OK) rope >=0.9.4 : 0.10.7 (OK) sphinx >=0.6.6 : 1.6.3 (OK) sympy >=0.7.3 : 1.1.1 (OK)

[bcolsen]

@ceaza What's not working exactly? Do you have an example of code that won't complete?

[stonebig]

is this a possible solution ? ("pip uninstall enum34" , as the module is wrongly required per some pymc3 wheels on python >= 3.4) https://github.com/spyder-ide/spyder/issues/5071#issue-253179855

[ceaza]

@stonebig @bcolsen removing enum34 did the trick.

[ewadaniela]

Auto-completion in Spyder works correctly if there are No white spaces in the project directory name.

[zkwabm]

@stonebig Yes! Removing enum34 does work!

[CAM-Gerlach]

FYI, just to keep things organized, per the most recent plans I'm aware of, most of these issues should finally be resolved for good with the Language Server work @andfoy is doing, currently pegged for Spyder 4 beta 2 if funding comes through. Also, similar: #5782

[ccordoba12]

This can't be closed yet because we could implement a new completion client that uses an IPython kernel behind the scenes.

[kaleb-keny]

Hello I wanted to raise an issue on this, but I see it was raised and closed... It's the same issue that the editor doesn't provide any code completion unlike the IPython on the right... I uninstalled kite, because it was bugging me with notifications... Was this closed because of kite?

[ccordoba12]

Please see my comment immediately above yours.

[mauricio-fernandez-l]

Please forgive me, if I am in the wrong issue or if my question is trivially solved. I do not understand how I can activate default autocompletion of local project modules. Autocompletion works fine in the console, once a module is imported, but not in the editor. I found a workaround by using

import sys
sys.path.append('.')

Do I always need to append the local project folder with sys.path.append('.') in order to allow in the editor for autocompletion of local project folders and modules or can I do this in a better manner? Its surely not a biggie, but I wanted to know if there is a better built-in way. Some of my system details:

• Windows 10
• Spyder 4.1.5 (installed through miniconda)
• Python 3.7.9
• IPython 7.18.1