search

jorgenschaefer / elpy

Emacs Python Development Environment
GNU General Public License v3.0
1.89k stars 260 forks source link

readline is not detected when running self tests #1443

Open sten0 opened 5 years ago

sten0 commented 5 years ago

Hi Jorgen,

We're 4.5 months away from Debian's freeze, so I've started to QA Elpy for the next stable release. The following is part of the console output of self-tests when building for Debian: Warning (python): Your ‘python-shell-interpreter’ doesn’t seem to support readline, yet ‘python-shell-completion-native-enable’ was t and "python" is not part of the ‘python-shell-completion-native-disabled-interpreters’ list. Native completions have been disabled locally.

I've confirmed this behaviour is present with configuration to use system-provided python3 (see #1442 for links to config). I guess the first question is: Would you prefer if Elpy on Debian defaults to a different interpreter such as ipython rather than /usr/bin/python3 -i? Debian apt-rdepends python2 and python3 both show libreadline7 in all of stretch, buster, and sid. python -vv outputs

# trying readline.x86_64-linux-gnu.so
# trying readline.so
# trying readlinemodule.so
# trying readline.py
# trying readline.pyc
# trying /usr/lib/python2.7/readline.x86_64-linux-gnu.so
# trying /usr/lib/python2.7/readline.so
# trying /usr/lib/python2.7/readlinemodule.so
# trying /usr/lib/python2.7/readline.py
# trying /usr/lib/python2.7/readline.pyc
# trying /usr/lib/python2.7/plat-x86_64-linux-gnu/readline.x86_64-linux-gnu.so
# trying /usr/lib/python2.7/plat-x86_64-linux-gnu/readline.so
# trying /usr/lib/python2.7/plat-x86_64-linux-gnu/readlinemodule.so
# trying /usr/lib/python2.7/plat-x86_64-linux-gnu/readline.py
# trying /usr/lib/python2.7/plat-x86_64-linux-gnu/readline.pyc
# trying /usr/lib/python2.7/lib-tk/readline.x86_64-linux-gnu.so
# trying /usr/lib/python2.7/lib-tk/readline.so
# trying /usr/lib/python2.7/lib-tk/readlinemodule.so
# trying /usr/lib/python2.7/lib-tk/readline.py
# trying /usr/lib/python2.7/lib-tk/readline.pyc
# trying /usr/lib/python2.7/lib-dynload/readline.x86_64-linux-gnu.so
dlopen("/usr/lib/python2.7/lib-dynload/readline.x86_64-linux-gnu.so", 2);
import readline # dynamically loaded from /usr/lib/python2.7/lib-dynload/readline.x86_64-linux-gnu.so

and python3 -vv confirms equivalent support, outputing

# trying /usr/lib/python3.5/readline.cpython-35m-x86_64-linux-gnu.so
# trying /usr/lib/python3.5/readline.abi3.so
# trying /usr/lib/python3.5/readline.so
# trying /usr/lib/python3.5/readline.py
# trying /usr/lib/python3.5/readline.pyc
# trying /usr/lib/python3.5/plat-x86_64-linux-gnu/readline.cpython-35m-x86_64-linux-gnu.so
# trying /usr/lib/python3.5/plat-x86_64-linux-gnu/readline.abi3.so
# trying /usr/lib/python3.5/plat-x86_64-linux-gnu/readline.so
# trying /usr/lib/python3.5/plat-x86_64-linux-gnu/readline.py
# trying /usr/lib/python3.5/plat-x86_64-linux-gnu/readline.pyc
# trying /usr/lib/python3.5/lib-dynload/readline.cpython-35m-x86_64-linux-gnu.so
# extension module 'readline' loaded from '/usr/lib/python3.5/lib-dynload/readline.cpython-35m-x86_64-linux-gnu.so'
# extension module 'readline' executed from '/usr/lib/python3.5/lib-dynload/readline.cpython-35m-x86_64-linux-gnu.so'
import 'readline' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7fb825c402e8>
import 'atexit' # <class '_frozen_importlib.BuiltinImporter'>

Steps to reproduce

https://ci.debian.net/data/autopkgtest/unstable/amd64/e/elpy/820104/log.gz

Experimental python3-only build has pretty much the same error, just s/python/python3/

I'm assuming the test for readline functionality actually tests the interface and isn't a simple pattern match. Please let me know what else I can do to help out.

Cheers, Nicholas

P.S. still waiting for yasnippets-snippets to merge that old PR...

jorgenschaefer commented 5 years ago

This is an upstream problem with Emacs, actually. See https://debbugs.gnu.org/cgi/bugreport.cgi?bug=25753#44

sten0 commented 5 years ago

Hi @jorgenschaefer !

I looked into this a bit more, and yes, I see what you mean, these strings come from Emacs and not from Elpy. I decided to dig into this issue some more, to find ways to cause Emacs to emit this warning.

This is an upstream problem with Emacs, actually. See https://debbugs.gnu.org/cgi/bugreport.cgi?bug=25753#44

I'm having trouble seeing how a macOS-specific bug (I skimmed all merged bugs too) caused by a Python default of libedit (BSD's editline) is related to Debian systems where Python is built against GNU readline7. Eg, digging further into python3 -vv output, the following Python readline module is used: 

ldd /usr/lib/python3.5/lib-dynload/readline.cpython-35m-x86_64-linux-gnu.so
        linux-vdso.so.1 (0x00007ffd560e5000)
        libreadline.so.7 => /lib/x86_64-linux-gnu/libreadline.so.7 (0x00007fdb8469b000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fdb8447e000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fdb840df000)
        libtinfo.so.5 => /lib/x86_64-linux-gnu/libtinfo.so.5 (0x00007fdb83eb5000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fdb84af0000)

Environment:

  1. Elpy's Python bits are purposefully only installed for Python 3 (Debian package).
  2. With proper configuration of rpc-python-command and python-shell-interpreter, C-c C-p works as expected in an interactive Emacs session, and native readline completion is correctly detected. eg: This configuration https://salsa.debian.org/emacsen-team/elpy/blob/master/debian/debian-autoloads.el produces the following result: 
    Python 3.5.3 (default, Jan 19 2017, 14:11:04) 
[GCC 6.3.0 20170118] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> python.el: native completion setup loaded
>>>
  3. I've also started setting elpy-rpc-pythonpath nil Debian's default configuration (also used for testing), but this is probably irrelevant, and probably just a somewhat paranoid expression of "reduce as many variables as possible!"

Here's how I was able to trigger doesn’t seem to support readline:

If everything at "2" doesn't match the Python module path, Emacs emits this cryptic--and I suspect misleading--warning. My suspicion is that this is another issue related to using Python 3 at /usr/bin/python3 and not system integration, because the warning is emitted both during Elpy's selftests (run from its source dir) and from the autopkgtests (run using the Elpy installed system-wide).

Given that it is Emacs and not Elpy emitting the warning when run in --batch mode, I'm not sure where to look in Elpy's source, so any pointers would be much appreciated. Maybe it would be a better use of time to wait until Emacs 26.1 is in Debian to see if the issue magically goes away or if the warning becomes more useful?

Cheers, Nicholas

jorgenschaefer commented 5 years ago

I'm afraid I have no idea how to tackle the issue, as I have not quite understood why Emacs emits those warnings in the first place. "Waiting and hoping it gets fixed" seems like the solution with the least effort required, but I'm not sure if that fits with your Debian timeline plans?