Feature request: function highlighting

[petobens]

Hi @tweekmonster I was wondering if it is possible to highlight imports of the form from foo import bar as functions rather than Keywords? For instance in the next image I would like bar to be highlighted as print.

If this is not possible or completely ridiculous please ignore this request. Thanks!!!!

[tweekmonster]

That's actually a good idea! It would be useful to know if the imported object is a function (or a class). I'll have to see what it'll take since this wouldn't be a trivial feature. The star_import.py script would need to be repurposed while avoiding the reinvention of Jedi.

[tweekmonster]

Unrelated: your screenshot shows that the import pandas as pd is incorrectly highlighted. There'll be a fix for that soon.

[petobens]

Awesome. Thank you!!!

[tweekmonster]

@petobens I got a something working. Unfortunately, it's pretty slow. If you're using Neovim, it will use async calls to parse the imports. I haven't gotten around to supporting Vim's new async calls, so I'm not merging it into master yet.

If you aren't using a version of Vim that supports async calls, you can enable the feature with let g:impsort_allow_slow_parse = 1. Depending on what imports are involved, there will be a noticeable (more than 1s) delay when editing the file.

There are two new highlight groups with this feature: pythonImportedFuncDef and pythonImportedClassDef. They're linked to Function and Type respectively.

[petobens]

Woww that was fast!!! I do have vim with async calls. I''ll wait a bit then for you to eventually add support for it (or eventually try it without it).

One non-related question: do you use neovim full time? do you fully recommend it (particularly for python) over vim? Thanks once again!

[tweekmonster]

I'm working on using Jedi since my AST parser is very naïve. It'll be faster, but not fast enough.

do you use neovim full time?

Yes. I've been using it for a little over a year now and I don't find it any less stable than using the latest versions of Vim. The developers are a friendly bunch if you want to ask questions in their Gitter channel. It also has some funding and a lot of active contributors, so it's not a toy project.

do you fully recommend it (particularly for python) over vim?

I have a bias, so take it with a grain of salt: Yes. deoplete.nvim, and deoplete-jedi have worked well for Python completions. deoplete-jedi (disclosure: I'm a collaborator) understands virtualenvs and works with pyenv. You would still want to install jedi-vim but with completions disabled.

Python is admittedly a little difficult to setup. If you want to give it a try, I wrote this wiki to help explain setting up Python for use with Neovim: https://github.com/tweekmonster/nvim-python-doctor/wiki

[petobens]

Awesome. Thank you very much for the explanation and links!

[tweekmonster]

@petobens The import-types branch has been updated and works with mainstream Vim's job control. Give it a try and let me know if you find any issues with it.

[petobens]

Hi @tweekmonster, I have python 3 installed (also python 2) as well as jedi. However when trying the new branch I get

[impsort] ImportError: No module named jedi

For the record which python returns /usr/bin/python and which python3 gives /usr/local/bin/python3. Finally echo has('python') returns 0 while echo has('python3') returns 1.

Please let me know if you need any further information. PS: I'm using vim-jedi and it seems to correctly detect the jedi module.

[petobens]

Also would it be possible to specify a cache directory with a global variable? Something like g:impsort_cache_dir? Thanks once again.

[petobens]

One last question: would it be possible to use async calls with neocomplete and jedi-vim or we need deoplete for that? There is an issue opened here.

[tweekmonster]

I have python 3 installed (also python 2) as well as jedi. However when trying the new branch I get

@petobens Sorry forgot to mention that. Jedi is added as a git submodule. You'll need to "reinstall" with your plugin manager. If that doesn't work, you'll need to run git submodule update --init. To be clear: you don't need to install Jedi yourself.

Also would it be possible to specify a cache directory with a global variable?

It wouldn't be difficult, but why do you want to change it? I'd rather not formalize command line argument parsing for the script if I don't need to.

would it be possible to use async calls with neocomplete and jedi-vim or we need deoplete for that?

For the purpose of completions, I think Vim's async features are too new. The docs are a draft and not complete. It gives the Vim devs room to adjust the API which could break plugins trying to use it today. impsort doesn't need the client/server aspect of it, so not much will be lost if API changes a bit. For vim-jedi, moving towards async will require them to create a client/server with the potential for breaking as well as maintaining backward compatibility. I don't see that happening soon.

deoplete is actually neocomplete's successor for Neovim. For the same reason mentioned above, I don't see async happening for that any time soon.

Using deoplete with vim-jedi as mentioned in that issue you linked isn't truly async. deoplete runs in a separate process, but calls vim-jedi's omnifunc to get completions. It prevents delays as you're typing, but the speed will depend on vim-jedi and the queued up completion requests would lead to delays before the completion popup is shown. This is something we dealt with in deoplete-jedi, but it was a non-trivial task.

[petobens]

WOWWWW this is just awesome!!!!! It seems to work pretty nicely. One minor question, consider the following screenshot:

import pandas as pd should be highligted as a pythonImportedObject or the behaviour has changed?

I'll report back if I encounter any bug. Thanks!!!!!

[petobens]

Not sure if a bug or not but in the following screenshot the time() function should not be highlighted as an import right? Maybe this is too difficult to implement:

[tweekmonster]

Not seeing what you're seeing.

I'm not sure what's up with the pandas import. Maybe a text change wasn't detected after adding the line. Try restarting Vim to see if it re-highlights it.

As for the double highlight in time.time() check to see if the impsortNonImport group exists. syntax list impsortNonImport

[petobens]

The impsortNonImport does exist but I still get the double highlight. Weird.

Regarding the pandas import can you try writing it as import pandas as pd instead of from pandas import pd. Thanks

[tweekmonster]

Sorry, I was rewriting the script for the screen shot and I typically don't alias imports except on from lines. But, the result is the same for me:

Do you see any errors when you run :messages ?

[petobens]

Mmmm I don't see any errors. However it is probably a problem at my end. I''l try to investigate a bit more and report back any finding.

[tweekmonster]

Here's a few things to try:

• See if b:impsort_highlight_job still exists after highlighting. That would prevent subsequent parsing.
• Run: echo b:python_imports. That contains a list of all of the found imported names. If pd isn't in there, it must mean that Jedi isn't finding it for some reason.
• Run echo impsort#get_all_imported(). This will show all the imported names, but parsed in Vim instead of Python. If it's not found here, it might not be included in the lines that are passed to the python script.

[petobens]

pd is found by impsort#get_all_imported() but not by b:python_imports so Jedi is not finding it for some reason (no idea why though). Do you know how can I check where is jedi looking for the pandas module?

[tweekmonster]

@petobens I figured out the problem. I guess there's an import and module type reported by Jedi. pd is a module, and I was only testing for import. It'll be fixed soon.

[petobens]

Yes, the pandas issue is indeed fixed. Thank you! Any idea why I still get the double highlight when using time? Maybe the following gif gives some hints?

[petobens]

Hi @tweekmonster I have one question that is probably totally outside from the scope of your plugin but maybe you can help me out. Can I somehow highlight every method as function? Is there a regex that I can add to the syntax highlighting?

For instance would like to highlight upper() as a function in the following code:

foo = 'foo'
bar = foo.upper()

Thanks for the help and once again sorry for asking this unrelated question.

[tweekmonster]

@petobens Sorry, I thought I responded to this but I guess I didn't. I'm afraid that is out of scope for impsort. But, you can highlight it yourself if you wanted to. Add this to after/syntax/python.vim:

syntax match pythonAttribute /\.\h\w*/hs=s+1

Or only for functions:

syntax match pythonAttribute /\.\h\w*(/hs=s+1,he=e-1

Then add something like this to your color scheme:

highlight default pythonAttribute ctermfg=red guifg=red

[petobens]

Thank you!!!!!