search

tkf / emacs-jedi

Python auto-completion for Emacs
http://tkf.github.io/emacs-jedi/latest/
666 stars 89 forks source link

[question] There's something wrong with jedi:install-server #343

Closed kirk86 closed 5 years ago

kirk86 commented 5 years ago

I'm using company and company-jedi. What I've noticed is that whenever I run jedi:install-server for some reason it brings a deprecated version of jediepcserver.py file where the deferred error of function signatures is still apparent. I've checked the master branch here and the jediepcserver.py has been corrected not to have those errors but still for some reason which I cannot explain the command jedi:install-server brings an old deprecated version of jediepcserver.py can someone have a look at?

I've also made emacs-company-jedi aware of this issue but I think the problem might lie here since in my understanding jedi:install-server brings or at least should bring the files of this repo but for some reason jediepcserver.py is outdated or deprecated with the deferred function signature errors still there in the file.

Here's my config info:

(use-package comapany
  :ensure t
  :init (global-company-mode t))

(use-package company-jedi
  :ensure t
  :init (add-to-list 'company-backends ' company-jedi))

(use-package python
  :hook ((python-mode . jedi:setup)))

jedi:show-version-info image

Here's the proof, this is jediepcserver.py just after executing jedi:install-server image

As you can see the jedi.api.defined_names should have been jedi.api.names instead.

Why is this happening, any ideas?

immerrr commented 5 years ago

That is interesting, let me try to reproduce it and get back to you.

In the meanwhile, obligatory question: could you please check that you are using the latest version of jedi-core?

kirk86 commented 5 years ago

image

Some other info

pip packages
jedi: 0.15.1
epc: 0.05
parso: 0.5.1
kirk86 commented 5 years ago

@immerrr I thinks I know what's happening jediepcserver.py seems to be different between melpa-stable and melpa. Although the one coming from melpa doesn't seem to have the deprecated calls to jedi.api.defined_names but rather to the correct ones jedi.api.names it still gives me deferred error whenever I call jedi:get-in-function-call. On the other hand jediepcserver.py coming from melpa stable has the deprecated calls to jedi.api.defined_names but after changing lines 105, 107, 179 it works fine whenever I call jedi:get-in-function-call. Could you please have a look and fix that in both melpa and melpa-stable?

immerrr commented 5 years ago

Yep, that's true, the stable distribution hasn't been updated in a while, so that could be the reason.

kirk86 commented 5 years ago

Sill though the melpa one gives me deferred error, the only way I could make ti work was to modify the melpa-stable one in the lines I mentioned above

immerrr commented 5 years ago

That is interesting... jedi-core package should contain a version of jediepcserver.py, and the one in melpa should be much fresher

immerrr commented 5 years ago

20191007.653 version on melpa should have this fixed, let me check...

kirk86 commented 5 years ago

I confirm that 20191007.653 does solve the problem but the whole melpa-stabe thing is becoming like a joke. I don't understand the point of calling it stable and confusing the users thinking they'll have a more stable experience there than in melpa when actually in contains outdated buggy code.

kirk86 commented 5 years ago

@immerrr Any ideas if this could be caused due to jedi backend.

immerrr commented 5 years ago

Not sure.. FWIW jedi (the python package) has been slow for me for a couple releases now, it is very noticeable for example in IPython input prompt.

immerrr commented 5 years ago

I confirm that 20191007.653 does solve the problem but the whole melpa-stabe thing is becoming like a joke

I understand your frustration, "bit rot" has not been gentle to jedi-core's stable release, and neither have I. I'll see if I can bump the stable release shortly. (#336 is a related issue)

kirk86 commented 5 years ago

jedi (the python package) has been slow for me for a couple releases now

Maybe worth reporting as bug to jedi? What other alternatives are there especially for python?

immerrr commented 5 years ago

You could try installing typeshed, which is a set of stubs for popular libraries out there. Type stubs reduce the amount of work Jedi has to do when deducing the correct types.

But in general, I don't see an open-source alternative in Python world. rope couple of years ago was already quite dated and not enjoying too much love from the community. PyCharm has a decent completion engine, but it comes with the PyCharm itself.

Oh, for Emacs itself, you could try youcompleteme, there's a ycmd server that provides completions for multiple languages, including Python via Jedi. It has a completely different binding, so could help if the problem is in emacs-jedi itself.

immerrr commented 5 years ago

Maybe worth reporting as bug to jedi?

Could work. It would help if you had a clear-cut completion example that did not require emacs or emacs-jedi, there's an example in Jedi docs on how to invoke completion from Python code, that would help narrowing down the scope of the sluggishness:

https://jedi.readthedocs.io/en/latest/docs/api.html

>>> import jedi
>>> source = '''
... import json
... json.lo'''
>>> script = jedi.Script(source, 3, len('json.lo'), 'example.py')
>>> script
<Script: 'example.py' ...>
>>> completions = script.completions()
kirk86 commented 5 years ago

Thanks for the ycmd pointer! Have you tried it yet? Regarding the jedi example still not sure what is that supposed to do other just showcasing jedi. There's no (as far as I understand) realistic benchmarking scenario to show how fast or slow jedi is.

PyCharm has a decent completion engine, but it comes with the PyCharm itself.

What are they using? Did they build their own or using something open-source?

immerrr commented 5 years ago

Thanks for the ycmd pointer! Have you tried it yet?

Nop, I'm on a "dogfooding" diet :)

Regarding the jedi example still not sure what is that supposed to do other just showcasing jedi. There's no (as far as I understand) realistic benchmarking scenario to show how fast or slow jedi is.

I mean, if you have a specific example that takes very long to complete, you could prepare a script for it: take source = open(path_to_source_file).read(), figure out the line and column numbers, and then run it in the terminal measuring the time spent. If it is still too slow, that's a Jedi issue.

If not, the issue could be with the jediepcserver. You could try enabling logging for the server, btw, if you haven't done it already. I have the following config:

jedi:server-args `("--log-level=DEBUG"
                      ,(format "--log=%s" (expand-file-name "~/jedi.log"))
                      "--log-rotate-max-size=1000000"
                      "--log-rotate-max-count=3"
                      "--log-traceback")

The log, although quite verbose, contains timestamps and could lead to some insights as to what is causing the slowness.

kirk86 commented 5 years ago

Thanks for this tip, it's actually very useful.

Now even after running jedi:install-server whenever I close the file and reopen it I get the following message

image

kirk86 commented 5 years ago

@immerrr Is this helpful?

Debugger entered--Lisp error: (wrong-type-argument epc:manager nil)
  signal(wrong-type-argument (epc:manager nil))
  jedi:epc--start-epc("/Users/jm/.emacs.d/.python-environments/default/bin/jediepcserver" ("--virtual-env" "/Users/jm/miniconda3/envs/tf" "--log-level=DEBUG" (\, (format "--log=%s" (expand-file-name "~/.emacs.d/jedi.log"))) "--log-rotate-max-size=1000000" "--log-rotate-max-count=3" "--log-traceback"))
  jedi:server-pool--start(("/Users/jm/.emacs.d/.python-environments/default/bin/jediepcserver" "--virtual-env" "/Users/jm/miniconda3/envs/tf" "--log-level=DEBUG" (\, (format "--log=%s" (expand-file-name "~/.emacs.d/jedi.log"))) "--log-rotate-max-size=1000000" "--log-rotate-max-count=3" "--log-traceback"))
  jedi:start-server()
  jedi:get-epc()
  jedi:call-deferred(complete)
  company-jedi-candidates(#f(compiled-function (candidates) #<bytecode 0x405934f1>))
  company--fetch-candidates("")
  company-calculate-candidates("" nil)
  company--begin-new()
  company--perform()
  company-auto-begin()
  company-idle-begin(#<buffer testpyth.py> #<window 1 on testpyth.py> 24 69)
  apply(company-idle-begin (#<buffer testpyth.py> #<window 1 on testpyth.py> 24 69))
  timer-event-handler([t 23968 47558 100681 nil company-idle-begin (#<buffer testpyth.py> #<window 1 on testpyth.py> 24 69) nil 0])
immerrr commented 5 years ago

Looks like a problem in the jedi:server-args variable, it has to be a flat list of strings and nothing else, for example mine is C-h v jedi:server-args:

("--log-level=DEBUG" "--log=/home/immerrr/jedi.log" "--log-rotate-max-size=1000000" "--log-rotate-max-count=3" "--log-traceback")

The string I sent you contained a comma which is a special syntax for evaluation inside backquotes and looks like it did not expand well.

kirk86 commented 5 years ago

Can I set it as quoted (setq jedi:server-args '(whatever))?

immerrr commented 5 years ago

Yes, a quoted list of strings '("foo" "bar" "baz") should work.

kirk86 commented 5 years ago

@immerrr This is what I have in my config image

I remove elpa, .python_environments and all *.elc

fire up emacs, installs packages

I use pyvenv and activate the environment which contains python jedi, epc, parso then M-x jedi:i-s

The C-c C-f testpython.py and this is what I get

Error (jedi): 
================================
Failed to start Jedi EPC server.
================================

*** EPC Error ***
Server may raise an error. Use "M-x epc:pop-to-last-server-process-buffer RET" to see full traceback:
usage: jediepcserver [-h] [--address ADDRESS] [--port PORT]
                     [--port-file PORT_FILE] [--sys-path SYS_PATH]
                     [--sys-path-append SYS_PATH_APPEND]
                     [--virtual-env VIRTUAL_ENV] [--log LOG]
                     [--log-level {CRITICAL,ERROR,WARN,INFO,DEBUG}]
                     [--log-rotate-max-size LOG_ROTATE_MAX_SIZE]
                     [--log-rotate-max-count LOG_ROTATE_MAX_COUNT]
                     [--log-traceback] [--pdb] [--ipdb]
jediepcserver: error: unrecognized arguments: /Users/jm/.emacs.d/jedi.log

*** EPC Server Output (last 10 lines) ***
usage: jediepcserver [-h] [--address ADDRESS] [--port PORT]
                     [--port-file PORT_FILE] [--sys-path SYS_PATH]
                     [--sys-path-append SYS_PATH_APPEND]
                     [--virtual-env VIRTUAL_ENV] [--log LOG]
                     [--log-level {CRITICAL,ERROR,WARN,INFO,DEBUG}]
                     [--log-rotate-max-size LOG_ROTATE_MAX_SIZE]
                     [--log-rotate-max-count LOG_ROTATE_MAX_COUNT]
                     [--log-traceback] [--pdb] [--ipdb]
jediepcserver: error: unrecognized arguments: /Users/jm/.emacs.d/jedi.log

*** EPC Server Config ***
Server arguments: ("/Users/jm/.emacs.d/.python-environments/default/bin/jediepcserver" "--virtual-env" "/Users/jm/miniconda3/envs/tf" "--log-level=DEBUG" "/Users/jm/.emacs.d/jedi.log" "--log-traceback")
Actual command: /Users/jm/.emacs.d/.python-environments/default/bin/jediepcserver
VIRTUAL_ENV envvar: "/Users/jm/miniconda3/envs/tf"

*** jedi-mode is disabled in #<buffer testpyth.py> ***
Fix the problem and re-enable it.

*** You may need to run "M-x jedi:install-server". ***
This could solve the problem especially if you haven't run the command yet
since Jedi.el installation or update and if the server complains about
Python module imports.

Now what's wrong with this, I don't know?

immerrr commented 5 years ago

In your server arguments

("/Users/jm/.emacs.d/.python-environments/default/bin/jediepcserver" "--virtual-env" "/Users/jm/miniconda3/envs/tf" "--log-level=DEBUG" "/Users/jm/.emacs.d/jedi.log" "--log-traceback")

instead of 

"/Users/jm/.emacs.d/jedi.log"

it should be 

"--log=/Users/jm/.emacs.d/jedi.log"

which is what jediepcserver is trying to tell by saying

error: unrecognized arguments: /Users/jm/.emacs.d/jedi.log

kirk86 commented 5 years ago

Oh crap I missed that, I was reading on half of the buffer on my 13' screen.

kirk86 commented 5 years ago

@immerrr I see a bunch of errors in the logs. I'm attaching the file here for closer inspection. There must be something going wrong here.

jedi.log

immerrr commented 5 years ago

Those are scary stacktraces, but they are mostly related to the known assertion errors raised while fetching docstrings:

2019-10-11 19:03:25,780:WARNING:jediepcserver:Cannot get docstring for completion <Completion: root_factors>

Note that they are marked as WARNING, not ERROR, which indicates that the expected function was performed at least to some extent (without docstrings).

immerrr commented 5 years ago

In the meantime you can get rid of the especially annoying timeouts by increasing company-async-timeout (the default is 2 seconds which is barely enough to generate the list of completions for numpy library, and that is with an SSD containing all the necessary jedi cache). I have the following in my python-mode hook:

(setq-local company-async-timeout 10)