elpy-goto-definition does not find definition in some cases

[dmalyuta]

Issue

elpy-goto-definition is able to go to system functions as well as to certain project-defined custom functions, however some project-defined functions cannot be found (No definition found is returned).

Steps to reproduce

1. Use the following setup for elpy in your ~/.emacs.d/init.el:

(use-package elpy
    :ensure t
    :config
    (elpy-enable))

My full init.el can be found here.

2. sudo apt-get install python-pip
3. sudo pip install jedi flake8 importmagic autopep8
4. cd ~ && git clone https://github.com/ethz-asl/kalibr
5. Open Emacs, open file ~/kalibr/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccCalibrator.py.
6. Place cursor at line 49 inside the substring addDesignVariables and do M-x elpy-goto-definition
7. This returns No definition found in the minibuffer, yet this function is defined in the IccSensors.py file, which is in the same directory and is imported in the __init__.py which is also in the same directory.

Versions

• Emacs: 24.5.1
• OS: Ubuntu 14.04
• Python: 2.7.6

Expected behaviour

elpy-goto-definition should be able to find and go to the definition of the addDesignVariables function (which is one of the several project-defined functions for which I get No definition found - you can find other examples in the IccCalibrator.py file by trying out M-x elpy-goto-definition on them).

[jorgenschaefer]

Hello, and thanks for the report! I'm afraid that Python is a rather dynamic language, which makes finding definitions a heuristic process rather than something clean and simple. The backends Elpy uses (Jedi or Rope) try their best, but they can not always find definitions. You can try and report this problem with Jedi – this particular case should be doable for that library.

I'm sorry I can not be of more help. Your bug report is really excellent, I would love to get more like these! :-)

[dmalyuta]

Thanks for replying, I was also thinking that this may be a Jedi problem - so I made an issue over there (davidhalter/jedi#880). Out of interest, what is then your workflow for Python in this context, if it is a large codebase and you cannot autocomplete definitions of certain methods? Do you use etags or something?

[jorgenschaefer]

I don't think I have a particular "workflow" there, unless you count "type the name" as a workflow. For dynamic languages like Python, JavaScript, Ruby etc. I see completions as a nice bonus, not something to expect and take for granted.

[dmalyuta]

I see - what are then the "primary" features that you expect when coding in Python? I'm asking because I'm only starting out with this language and you have a lot of experience with it (enough to have written a great Emacs package for it!) - so I'd like to know what professional developers consider as relevant features.

[jorgenschaefer]

I suspect you come from a Java+IDE background or similar, and Python is your first dynamic language?

Important features tend to rely on: Syntax highlighting and coding style notifications (flake8), primarily. Being able to run tests quickly and easily (C-c C-t and M-x recompile). Being able to quickly grep/ag through the whole project. Completions and "go to definition" are nice, but as I said, for me it's more that I'm happy when they work and just deal with it when they don't.

[dmalyuta]

I come from C++ with Eclipse, more recently C++ with Emacs, background - so yes, I've got a mindset of expecting auto-completion to work flawlessly. Have you had any good/bad experiences using Emacs' etags for Python?

But I see what you mean - thanks for your answers! Hopefully the issue I posted on the Jedi github page will at some point be resolved.

[jorgenschaefer]

Have you had any good/bad experiences using Emacs' etags for Python?

I have no good experiences using any *tags with any language, really. :-)

[dmalyuta]

Ok, thank you very much for your answers.

[analyticd]

This is closed but I'll just add that I use a technique in such cases of enlisting the help of ag, but limited to my virtual env (for the punchline go to the gist link at the end):

(require 'ag)
  (after 'my-ag-config  ; my ag config is at the bottom of this post; after is just a macro for eval-after-load
    (defun my-python-jump-to-definition (string directory)
      "Search using ag in a given DIRECTORY for a given literal search STRING,
with STRING defaulting to the symbol under point."
      (interactive (list (ag/read-from-minibuffer "Search string")
                        (read-directory-name
                          "Directory: "
                          (concat python-shell-virtualenv-root
                                  "/"
                                  "/lib/python3.6/site-packages/"))))
      (ag/search string directory)))

and then bind that via

(bind-key "M-C-." 'my-python-jump-to-definition elpy-mode-map)

When invoked, I typically alter the search string to, say, def mean(, instead of just mean, then I limit what library I want to search in, say pandas, by typing in the library where my own analysis or understanding of Python libs says it is likely to exist (again, because Python is dynamic, you have to do a bit more reasoning to trace things out), then hit enter. I've A/B'd this technique against using PyCharm and I find that on average, this is actually better when elpy goto definition fails (which I have bound to M-.).

Another individual does it like this (and I use this too):

(defun goto-def-or-rgrep ()
      "Go to definition of thing at point or do an rgrep in
      project if that fails"
      (interactive)
      (condition-case nil (elpy-goto-definition)
        (error (elpy-rgrep-symbol (thing-at-point 'symbol)))))
  (bind-key "M-." 'goto-def-or-rgrep elpy-mode-map) ; bind-key comes with use-package

But that only will search in the project directory (as determined by elpy - see source of elpy-rgrep-symbol). You can customize elpy to more broadly define your project directories by defining a new function that would return also system libraries you need to jump to def on. However, I find that my my-python-jump-to-definition works great as the ag emacs package displays things nicely.

My ag config (referenced above):

(use-package ag
  :ensure t
  ;; :commands (ag ag-mode ag-files ag-regexp)
  :bind ("C-c u s" . ag)
  :init
  (setq ag-highlight-search t)
  (setq ag-reuse-buffers t)
  (defun my-setup-ag ()
    "Function called to set my ag stuff up."
    (toggle-truncate-lines t)
    (switch-to-buffer-other-window "*ag search*"))
  (add-hook 'ag-mode-hook 'my-setup-ag)
  (after 'evil
    (evil-set-initial-state 'ag-mode 'normal))
  :config
  (unbind-key "k" ag-mode-map)   ;; unbind kill window
  (unbind-key "h" ag-mode-map)
  (after 'evil
    (evil-define-key 'normal ag-mode-map (kbd "k") 'nil)
    ;; (evil-define-key 'motion ag-mode-map (kbd "k") 'compilation-previous-error)
    ;; (evil-define-key 'motion ag-mode-map (kbd "j") 'compilation-next-error)
    ))

Finally here is the after macro referenced above:

(defmacro after (feature &rest body)
  "After FEATURE is loaded, evaluate BODY."
  (declare (indent defun))
  `(eval-after-load ,feature
    '(progn ,@body)))

And once I jump to a location either via elpy-goto-definition or my-python-jump-to-definition I then employ idomenu (emacs package) to jump to classes and methods. idomenu is a lot smarter in python than a lot of people realize. If multiple classes are defined in one file, say in pandas somewhere to where you have jumped, then when you invoke idomenu (which I bind to evil-leader,i for ease of use) it will first list all classes in the file such that when you select one, you can then subsequently choose from another method of everything to do with your class choice.

Oh, and to make this all work I put my Anaconda Python root in my path, i.e., ~/anaconda/bin, and then I use the conda emacs package (not anaconda-mode) to manage my virtualenv situation which was previously created with Anaconda via conda env create (see this helpful blog post, not by me, http://tdhopper.com/blog/2015/Nov/24/my-python-environment-workflow-with-conda/ for more information on this). I find conda necessary because it uses Anaconda's environment.yml file in your project root to determine which Python process to invoke, elpy doesn't do this. To make it easier on future readers, here I recapitulate some of above and include my total python config at the following gist:

https://gist.github.com/analyticd/cfb253cce2ab5c9b97aed7def1e37a04

[avatar-lavventura]

@analyticd I am having following error with your code:

Debugger entered--Lisp error: (void-function after)
  (after (quote my-ag-config) (defalias (quote my-python-jump-to-definition) (function (lambda (string directory)
(interactive (list (ag/read-from-minibuffer "Search string") (read-directory-name "Directory: " (concat
python-shell-virtualenv-root "/" "/lib/python3.7/site-packages/")))) (ag/search string directory)))))
  eval-buffer(#<buffer  *load*> nil "/home/alper/.emacs" nil t)  ; Reading at buffer position 105239
  load-with-code-conversion("/home/alper/.emacs" "/home/alper/.emacs" t t)
  load("~/.emacs" t t)
  #f(compiled-function () #<bytecode 0x1e3119>)()
  command-line()
  normal-top-level()

[analyticd]

Hi Alper,

I wasn't able to find issue #1087? Were you able to resolve the issue you were experiencing with my code? I'd like to help you if possible. Perhaps you have already resolved it since the issue appears to be closed, I am not sure. Anyway, I haven't used elpy for a long time, using python language server these days in Emacs. But from the stack trace it looks like you may need the after macro:

(defmacro after (feature &rest body) "After FEATURE is loaded, evaluate BODY." (declare (indent defun)) `(eval-after-load ,feature '(progn ,@body)))

I hope this helps. Please let me know if you need additional assistance and I will try to help.

Sincerely, @analyticd On Sat, Feb 13 2021, Alper Alimoglu notifications@github.com wrote:

@analyticd I am having following error with your code:

Debugger entered--Lisp error: (void-function after)
  (after (quote my-ag-config) (defalias (quote my-python-jump-to-definition) (function (lambda (string directory)
(interactive (list (ag/read-from-minibuffer "Search string") (read-directory-name "Directory: " (concat
python-shell-virtualenv-root "/" "/lib/python3.7/site-packages/")))) (ag/search string directory)))))
  eval-buffer(#<buffer  *load*> nil "/home/alper/.emacs" nil t)  ; Reading at buffer position 105239
  load-with-code-conversion("/home/alper/.emacs" "/home/alper/.emacs" t t)
  load("~/.emacs" t t)
  #f(compiled-function () #<bytecode 0x1e3119>)()
  command-line()
  normal-top-level()

[avatar-lavventura]

@analyticd Thanks for your reply. I have tried to apply your code mentioned in this issue, but faced with the error I mentioned on my previous comment. Should I make any change in your code to make it work?

(require 'ag)
  (after 'my-ag-config  ; my ag config is at the bottom of this post; after is just a macro for eval-after-load
    (defun my-python-jump-to-definition (string directory)
      "Search using ag in a given DIRECTORY for a given literal search STRING,
with STRING defaulting to the symbol under point."
      (interactive (list (ag/read-from-minibuffer "Search string")
                        (read-directory-name
                          "Directory: "
                          (concat python-shell-virtualenv-root
                                  "/"
                                  "/lib/python3.6/site-packages/"))))
      (ag/search string directory)))

---

I can open a new question or give more information if its required.

[analyticd]

I think that code I wrote is pretty crufty. It worked at the time. Jump to definition is something I rely on heavily and it was sort of flaky with elpy. Sometimes I had to rely on packages like dumbjump.

Moving forward to today, I have left elpy behind altogether and use the following which works pretty well for me:

Install https://github.com/palantir/python-language-server: pip install python-language-server

Then here are relevant bits of my config:

;; Package `lsp-mode' is an Emacs client for the Language Server ;; Protocol https://langserver.org/. It is where we get all of our ;; information for completions, definition location, documentation, ;; and so on. (use-package lsp-mode :config

;; Configure lsp-pyls (setq lsp-pyls-plugins-autopep8-enabled nil lsp-pyls-plugins-pycodestyle-enabled nil lsp-pyls-plugins-jedi-completion-enabled nil ;; See https://pypi.org/project/python-language-server/ about ;; default configurations. ;; Setting lsp-pyls-plugins-flake8-enabled t causes lsp-pyls to use ;; the [flake8] section of setup.cfg rather than [pycodestyle] ;; section. Note however, that for whatever reason, pyls picks up ;; max-line-length from [pycodestyle] section even though the ;; ignores are coming from [flake8]. It may be that that would ;; change if we explicitly disable pycodestyle as a pyls ;; configuration source. lsp-pyls-configuration-sources ["flake8"] lsp-pyls-plugins-flake8-enabled t )

;; Use the pyls-mypy plugin for live type checking. ;; Source: https://github.com/emacs-lsp/lsp-mode/pull/1317 ;; To investigate all client settings do C-h v lsp-client-settings (lsp-register-custom-settings '(("pyls.plugins.pyls_mypy.enabled" t t) ("pyls.plugins.pyls_mypy.live_mode" nil t)))

)

;; Dumbjump just in case nothing else works for jump to definition (use-feature dumb-jump :init (setq ;; dumb-jump-prefer-searcher 'rg dumb-jump-force-searcher 'rg dumb-jump-selector 'ivy) :config (add-hook 'xref-backend-functions #'dumb-jump-xref-activate))

;; eglot is better than microsoft's python language server client in my exerience (use-package eglot :after lsp :config (add-hook 'python-mode-hook 'eglot-ensure) (add-to-list 'eglot-server-programs '(python-mode . ("pyls"))))

I realize this didn't answer your question directly, but this is what I use now and it is so much better than the elpy days, jedi, rope or whatever days for me. I hope this helps.

On Mon, Feb 15 2021, Alper Alimoglu notifications@github.com wrote:

@analyticd Thanks for your reply. I have tried to apply your code mentioned in this issue, but faced with the error I mentioned on my previous comment. Should I make any change in your code to make it work?

(require 'ag)
  (after 'my-ag-config  ; my ag config is at the bottom of this post; after is just a macro for eval-after-load
    (defun my-python-jump-to-definition (string directory)
      "Search using ag in a given DIRECTORY for a given literal search STRING,
with STRING defaulting to the symbol under point."
      (interactive (list (ag/read-from-minibuffer "Search string")
                        (read-directory-name
                          "Directory: "
                          (concat python-shell-virtualenv-root
                                  "/"
                                  "/lib/python3.6/site-packages/"))))
      (ag/search string directory)))

---

I can open a new question or give more information if its required.

[avatar-lavventura]

@analyticd When I tried your code now I am getting following error message :( But I got the idea I will try to integrate python-language-server into emacs somehow.

Debugger entered--Lisp error: (void-function use-feature)
  (use-feature dumb-jump :init (setq dumb-jump-force-searcher (quote rg) dumb-jump-selector (quote ivy)) :config (add-hook (quote
xref-backend-functions) (function dumb-jump-xref-activate)))
  eval-buffer(#<buffer  *load*> nil "/home/alper/.emacs" nil t)  ; Reading at buffer position 106228
  load-with-code-conversion("/home/alper/.emacs" "/home/alper/.emacs" t t)
  load("~/.emacs" t t)
  #f(compiled-function () #<bytecode 0x1e3119>)()
  command-line()
  normal-top-level()

[Hi-Angel]

@avatar-lavventura I think it's a typo in the original post, it likely was supposed to be (use-package dumb-jump ….

Also make sure that if python code root is different from that of the project root, you add the subroot with (lsp-workspace-folders-add) command. To avoid missing definitions and stuff.