search

jupyter-xeus / xeus-python-wheel

Building a PyPI wheel for xeus-python
BSD 3-Clause "New" or "Revised" License
12 stars 13 forks source link

ModuleNotFoundError: No module named 'encodings' #28

Closed pmav99 closed 4 years ago

pmav99 commented 4 years ago

Tried to test the wheel package on Archlinux. Installation succeeds but running the kernel fails.

Traceback when I tried to create a new notebook ``` [I 21:09:27.173 LabApp] Creating new notebook in / [E 21:09:27.479 LabApp] Failed to run command: ['xpython', '-f', '/home/username/.local/share/jupyter/runtime/kernel-5f393192-24a0-40c0-9929-50967071da9e.json'] PATH='/home/username/.local/my_bin:/home/username/.local/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/lib/jvm/default/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl' with kwargs: {'stdin': -1, 'stdout': None, 'stderr': None, 'cwd': '/home/username/Prog/experiments/xeus', 'start_new_session': True} [E 21:09:27.479 LabApp] Uncaught exception POST /api/sessions?1585422567464 (::1) HTTPServerRequest(protocol='http', host='localhost:8888', method='POST', uri='/api/sessions?1585422567464', version='HTTP/1.1', remote_ip='::1') Traceback (most recent call last): File "/home/username/Prog/experiments/xeus/v38/lib/python3.8/site-packages/tornado/web.py", line 1703, in _execute result = await result File "/home/username/Prog/experiments/xeus/v38/lib/python3.8/site-packages/tornado/gen.py", line 742, in run yielded = self.gen.throw(*exc_info) # type: ignore File "/home/username/Prog/experiments/xeus/v38/lib/python3.8/site-packages/notebook/services/sessions/handlers.py", line 69, in post model = yield maybe_future( File "/home/username/Prog/experiments/xeus/v38/lib/python3.8/site-packages/tornado/gen.py", line 735, in run value = future.result() File "/home/username/Prog/experiments/xeus/v38/lib/python3.8/site-packages/tornado/gen.py", line 742, in run yielded = self.gen.throw(*exc_info) # type: ignore File "/home/username/Prog/experiments/xeus/v38/lib/python3.8/site-packages/notebook/services/sessions/sessionmanager.py", line 88, in create_session kernel_id = yield self.start_kernel_for_session(session_id, path, name, type, kernel_name) File "/home/username/Prog/experiments/xeus/v38/lib/python3.8/site-packages/tornado/gen.py", line 735, in run value = future.result() File "/home/username/Prog/experiments/xeus/v38/lib/python3.8/site-packages/tornado/gen.py", line 742, in run yielded = self.gen.throw(*exc_info) # type: ignore File "/home/username/Prog/experiments/xeus/v38/lib/python3.8/site-packages/notebook/services/sessions/sessionmanager.py", line 100, in start_kernel_for_session kernel_id = yield maybe_future( File "/home/username/Prog/experiments/xeus/v38/lib/python3.8/site-packages/tornado/gen.py", line 735, in run value = future.result() File "/home/username/Prog/experiments/xeus/v38/lib/python3.8/site-packages/tornado/gen.py", line 209, in wrapper yielded = next(result) File "/home/username/Prog/experiments/xeus/v38/lib/python3.8/site-packages/notebook/services/kernels/kernelmanager.py", line 168, in start_kernel super(MappingKernelManager, self).start_kernel(**kwargs) File "/home/username/Prog/experiments/xeus/v38/lib/python3.8/site-packages/jupyter_client/multikernelmanager.py", line 158, in start_kernel km.start_kernel(**kwargs) File "/home/username/Prog/experiments/xeus/v38/lib/python3.8/site-packages/jupyter_client/manager.py", line 305, in start_kernel self.kernel = self._launch_kernel(kernel_cmd, **kw) File "/home/username/Prog/experiments/xeus/v38/lib/python3.8/site-packages/jupyter_client/manager.py", line 212, in _launch_kernel return launch_kernel(kernel_cmd, **kw) File "/home/username/Prog/experiments/xeus/v38/lib/python3.8/site-packages/jupyter_client/launcher.py", line 135, in launch_kernel proc = Popen(cmd, **kwargs) File "/usr/lib/python3.8/subprocess.py", line 854, in __init__ self._execute_child(args, executable, preexec_fn, close_fds, File "/usr/lib/python3.8/subprocess.py", line 1702, in _execute_child raise child_exception_type(errno_num, err_msg, err_filename) FileNotFoundError: [Errno 2] No such file or directory: 'xpython' [W 21:09:27.482 LabApp] Unhandled error [E 21:09:27.483 LabApp] { "Host": "localhost:8888", "User-Agent": "Mozilla/5.0 (X11; Linux x86_64; rv:74.0) Gecko/20100101 Firefox/74.0", "Accept": "*/*", "Accept-Language": "en-US,en;q=0.5", "Accept-Encoding": "gzip, deflate", "Referer": "http://localhost:8888/lab/workspaces/auto-P", "Content-Type": "application/json", "Authorization": "token ae9b0e57efbe951d548a2f0e402396b7b302d4c3866bc079", "X-Xsrftoken": "2|90b5775f|5190b9ecbede45e1487497d9e06851c3|1584968398", "Origin": "http://localhost:8888", "Content-Length": "97", "Dnt": "1", "Connection": "keep-alive", "Cookie": "username-localhost-8888=\"2|1:0|10:1585422567|23:username-localhost-8888|44:NDY2OGNlMDA0YTZkNDkwMjkwZjcxMGUxNDViMTQwOWY=|f7a149c7508394d43982bcf415270d76d900093858163e8eedcd99348de1c0ec\"; _xsrf=2|90b5775f|5190b9ecbede45e1487497d9e06851c3|1584968398", "Pragma": "no-cache", "Cache-Control": "no-cache" } ```

What is strange is that even though the error is FileNotFoundError: [Errno 2] No such file or directory: 'xpython', the xpython binary does exist in the bin directory of the virtualenv. Executing that though, results in:

Traceback when I tried to execute "v38/bin/xpython" ``` registering handler for SIGSEGV PYTHONHOME set to /home/username/Prog/experiments/xeus/v38/bin/../ Python sys.path set to :/home/username/Prog/experiments/xeus/v38/bin/..//lib/python38.zip:/home/username/Prog/experiments/xeus/v38/bin/..//lib/python3.8:/home/username/Prog/experiments/xeus/v38/bin/..//lib/python3.8/lib-dynload:/home/username/Prog/experiments/xeus/v38/bin/..//lib/python3.8/site-packages:/home/username/Prog/experiments/xeus/v38/bin/..//../lb/python38.zip:/home/username/Prog/experiments/xeus/v38/bin/..//../lib/python3.8:/home/username/Prog/experiments/xeus/v38/bin/..//../lib/python3.8/lib-dynload:/home/username/Prog/experiments/xeus/v38/bin/..//../lib/python3.8/site-packages:/home/username/Prog/experiments/xeus/v38/bin/..//lib64/python38.zip:/home/username/Prog/experiments/xeus/v38/bin/..//lib64/python3.8:/home/username/Prog/experiments/xeus/v38/bin/..//lib64/python3.8/lib-dynload:/home/username/Prog/experiments/xeus/v38/bin/..//lib64/python3.8/site-packages:/home/username/Prog/experiments/xeus/v38/bin/..//../lib64/python38.zip:/home/username/Prog/experiments/xeus/v38/bin/..//../lib64/python3.8:/home/fe(5��� Python path configuration: PYTHONHOME = '/home/username/Prog/experiments/xeus/v38/bin/../' PYTHONPATH = (not set) program name = 'python3' isolated = 0 environment = 1 user site = 1 import site = 1 sys._base_executable = '/usr/bin/python3' sys.base_prefix = '' sys.base_exec_prefix = '' sys.executable = '/usr/bin/python3' sys.prefix = '' sys.exec_prefix = '' sys.path = [ '', '/home/username/Prog/experiments/xeus/v38/bin/..//lib/python38.zip', '/home/username/Prog/experiments/xeus/v38/bin/..//lib/python3.8', '/home/username/Prog/experiments/xeus/v38/bin/..//lib/python3.8/lib-dynload', '/home/username/Prog/experiments/xeus/v38/bin/..//lib/python3.8/site-packages', '/home/username/Prog/experiments/xeus/v38/bin/..//../lb/python38.zip', '/home/username/Prog/experiments/xeus/v38/bin/..//../lib/python3.8', '/home/username/Prog/experiments/xeus/v38/bin/..//../lib/python3.8/lib-dynload', '/home/username/Prog/experiments/xeus/v38/bin/..//../lib/python3.8/site-packages', '/home/username/Prog/experiments/xeus/v38/bin/..//lib64/python38.zip', '/home/username/Prog/experiments/xeus/v38/bin/..//lib64/python3.8', '/home/username/Prog/experiments/xeus/v38/bin/..//lib64/python3.8/lib-dynload', '/home/username/Prog/experiments/xeus/v38/bin/..//lib64/python3.8/site-packages', '/home/username/Prog/experiments/xeus/v38/bin/..//../lib64/python38.zip', '/home/username/Prog/experiments/xeus/v38/bin/..//../lib64/python3.8', '/home/username/Prog/experiments/xeus/v38/bin/..//../lib64/python3.8/lib-dynload', '/home/username/Prog/experiments/xeus/v38/bin/..//../lib64/python3.8/site-packages', '', ] Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding Python runtime state: core initialized ModuleNotFoundError: No module named 'encodings' Current thread 0x00007f6a896b6fc0 (most recent call first): ```

I also tried with Python 3.6 and Python 3.7 with the same errors:

Traceback when I tried to execute "v36/bin/xpython" ``` registering handler for SIGSEGV PYTHONHOME set to /home/username/Prog/experiments/xeus/v36/bin/../ Python sys.path set to :/home/username/Prog/experiments/xeus/v36/bin/..//lib/python36.zip:/home/username/Prog/experiments/xeus/v36/bin/..//lib/python3.6:/home/username/Prog/experiments/xeus/v36/bin/..//lib/python3.6/lib-dynload:/home/username/Prog/experiments/xeus/v36/bin/..//lib/python3.6/site-packages:/home/username/Prog/experiments/xeus/v36/bin/..//../lb/python36.zip:/home/username/Prog/experiments/xeus/v36/bin/..//../lib/python3.6:/home/username/Prog/experiments/xeus/v36/bin/..//../lib/python3.6/lib-dynload:/home/username/Prog/experiments/xeus/v36/bin/..//../lib/python3.6/site-packages:/home/username/Prog/experiments/xeus/v36/bin/..//lib64/python36.zip:/home/username/Prog/experiments/xeus/v36/bin/..//lib64/python3.6:/home/username/Prog/experiments/xeus/v36/bin/..//lib64/python3.6/lib-dynload:/home/username/Prog/experiments/xeus/v36/bin/..//lib64/python3.6/site-packages:/home/username/Prog/experiments/xeus/v36/bin/..//../lib64/python36.zip:/home/username/Prog/experiments/xeus/v36/bin/..//../lib64/python3.6:/home/feHyA=� Fatal Python error: Py_Initialize: Unable to get the locale encoding ModuleNotFoundError: No module named 'encodings' Current thread 0x00007f5aa65defc0 (most recent call first): [1] 59653 abort (core dumped) v36/bin/xpython ```
Traceback when I tried to execute "v37/bin/xpython" ``` registering handler for SIGSEGV PYTHONHOME set to /home/username/Prog/experiments/xeus/v37/bin/../ Python sys.path set to :/home/username/Prog/experiments/xeus/v37/bin/..//lib/python37.zip:/home/username/Prog/experiments/xeus/v37/bin/..//lib/python3.7:/home/username/Prog/experiments/xeus/v37/bin/..//lib/python3.7/lib-dynload:/home/username/Prog/experiments/xeus/v37/bin/..//lib/python3.7/site-packages:/home/username/Prog/experiments/xeus/v37/bin/..//../lb/python37.zip:/home/username/Prog/experiments/xeus/v37/bin/..//../lib/python3.7:/home/username/Prog/experiments/xeus/v37/bin/..//../lib/python3.7/lib-dynload:/home/username/Prog/experiments/xeus/v37/bin/..//../lib/python3.7/site-packages:/home/username/Prog/experiments/xeus/v37/bin/..//lib64/python37.zip:/home/username/Prog/experiments/xeus/v37/bin/..//lib64/python3.7:/home/username/Prog/experiments/xeus/v37/bin/..//lib64/python3.7/lib-dynload:/home/username/Prog/experiments/xeus/v37/bin/..//lib64/python3.7/site-packages:/home/username/Prog/experiments/xeus/v37/bin/..//../lib64/python37.zip:/home/username/Prog/experiments/xeus/v37/bin/..//../lib64/python3.7:/home/fe8^�K� Fatal Python error: initfsencoding: Unable to get the locale encoding ModuleNotFoundError: No module named 'encodings' Current thread 0x00007fa632be6fc0 (most recent call first): [1] 59717 abort (core dumped) v37/bin/xpython ```

PS. Please let me know if you want these tickets here, or you prefer to have them on the main repo.

pmav99 commented 4 years ago

For the record, I just tried this with python 3.7 in a debian box and it does give the same error:

Traceback from executing "venv/bin/xpython" ``` registering handler for SIGSEGV PYTHONHOME set to /home/panos/venv/bin/../ Python sys.path set to :/home/panos/venv/bin/..//lib/python37.zip:/home/panos/venv/bin/..//lib/python3.7:/home/panos/venv/bin/..//lib/python3.7/lib-dynload:/home/panos/venv/bin/..//lib/python3.7/site-packages:/home/panos/venv/bin/..//../lb/python37.zip:/home/panos/venv/bin/..//../lib/python3.7:/home/panos/venv/bin/..//../lib/python3.7/lib-dynload:/home/panos/venv/bin/..//../lib/python3.7/site-packages:/home/panos/venv/bin/..//lib64/python37.zip:/home/panos/venv/bin/..//lib64/python3.7:/home/panos/venv/bin/..//lib64/python3.7/lib-dynload:/home/panos/venv/bin/..//lib64/python3.7/site-packages:/home/panos/venv/bin/..//../lib64/python37.zip:/home/panos/venv/bin/..//../lib64/python3.7:/home/panos/venv/bin/..//../lib64/python3.7/lib-dynload:/home/panos/venv/bin/..//../lib64/python3.7/site-packages: Fatal Python error: initfsencoding: Unable to get the locale encoding ModuleNotFoundError: No module named 'encodings' Current thread 0x00007f53a0332bc0 (most recent call first): Aborted ```
SylvainCorlay commented 4 years ago

Thanks for reporting

FYI, the PyPI wheels are still experimental, and we recommend using a package manager, such as conda. We maintain conda-forge packages for xeus-python.

However, we are working hard on streamlining the wheels. At the moment, we have not bee working with virtual environments yet, but I am curious about the values of sys.prefix and sys.path in your case.

pmav99 commented 4 years ago

we recommend using a package manager, such as conda

Thank you for the suggestion, but I am afraid conda is not really an option for my usecase.

for venv in v36 v37 v38;
do
  echo "${venv}";
  echo -n 'prefix: '
  "${venv}"/bin/python -c 'import sys; print(sys.prefix)'
  echo -n 'path  : ' 
  "${venv}"/bin/python -c 'import sys; print(sys.path)'
  echo
done

v36
prefix: /home/username/Prog/experiments/xeus/v36
path  : ['', '/usr/lib/python36.zip', '/usr/lib/python3.6', '/usr/lib/python3.6/lib-dynload', '/home/username/Prog/experiments/xeus/v36/lib/python3.6/site-packages']

v37
prefix: /home/username/Prog/experiments/xeus/v37
path  : ['', '/usr/lib/python37.zip', '/usr/lib/python3.7', '/usr/lib/python3.7/lib-dynload', '/home/username/Prog/experiments/xeus/v37/lib/python3.7/site-packages']

v38
prefix: /home/username/Prog/experiments/xeus/v38
path  : ['', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/username/Prog/experiments/xeus/v38/lib/python3.8/site-packages']

And these are from the debian machine:

venv/bin/python -c 'import sys; print(sys.prefix)'
/home/panos/venv
venv/bin/python -c 'import sys; print(sys.path)'
['', '/usr/local/lib/python37.zip', '/usr/local/lib/python3.7', '/usr/local/lib/python3.7/lib-dynload', '/home/panos/venv/lib/python3.7/site-packages']

AFAIK python has been compiled in that machine, i.e. it has not been installed from a deb package.

joshbode commented 4 years ago

My temporary hack solution, that seems to work:

$ ls /usr/lib/python3.8 | grep -v site-packages | while read X; do ln -s /usr/lib/python3.8/$X ${VIRTUAL_ENV}/lib/python3.8/$X; done

after which I get:

$ jupyter console --kernel=xpython
registering handler for SIGSEGV
PYTHONHOME set to /home/josh/.virtualenvs/default-3.8/bin/../
Python sys.path set to :/home/josh/.virtualenvs/default-3.8/bin/..//lib/python38.zip:/home/josh/.virtualenvs/default-3.8/bin/..//lib/python3.8:/home/josh/.virtualenvs/default-3.8/bin/..//lib/python3.8/lib-dynload:/home/josh/.virtualenvs/default-3.8/bin/..//lib/python3.8/site-packages:/home/josh/.virtualenvs/default-3.8/bin/..//../lb/python38.zip:/home/josh/.virtualenvs/default-3.8/bin/..//../lib/python3.8:/home/josh/.virtualenvs/default-3.8/bin/..//../lib/python3.8/lib-dynload:/home/josh/.virtualenvs/default-3.8/bin/..//../lib/python3.8/site-packages:/home/josh/.virtualenvs/default-3.8/bin/..//lib64/python38.zip:/home/josh/.virtualenvs/default-3.8/bin/..//lib64/python3.8:/home/josh/.virtualenvs/default-3.8/bin/..//lib64/python3.8/lib-dynload:/home/josh/.virtualenvs/default-3.8/bin/..//lib64/python3.8/site-packages:/home/josh/.virtualenvs/default-3.8/bin/..//../lib64/python38.zip:/home/josh/.virtualenvs/default-3.8/bin/..//../lib64/python3.8:/home/josh/.virtualenvs/default-3.8/bin/..//../lib64/python3.8/lib-dynload:/home/josh/.virtu(B
Starting xeus-python kernel...

If you want to connect to this kernel from an other client, you can use the /home/josh/.local/share/jupyter/runtime/kernel-890598.json file.
Run with XEUS 0.23.9

In [1]:
joshbode commented 4 years ago

BTW, the problem appears to be that the sys.path does not include the base system paths:

Python path configuration:
  PYTHONHOME = '/home/josh/.virtualenvs/default-3.8/bin/../'
  PYTHONPATH = (not set)
  program name = 'python3'
  isolated = 0
  environment = 1
  user site = 1
  import site = 1
  sys._base_executable = '/home/josh/.virtualenvs/default-3.8/bin/python3'
  sys.base_prefix = ''
  sys.base_exec_prefix = ''
  sys.executable = '/home/josh/.virtualenvs/default-3.8/bin/python3'
  sys.prefix = ''
  sys.exec_prefix = ''
  sys.path = [
    '',
    '/home/josh/.virtualenvs/default-3.8/bin/..//lib/python38.zip',
    '/home/josh/.virtualenvs/default-3.8/bin/..//lib/python3.8',
    '/home/josh/.virtualenvs/default-3.8/bin/..//lib/python3.8/lib-dynload',
    '/home/josh/.virtualenvs/default-3.8/bin/..//lib/python3.8/site-packages',
    '/home/josh/.virtualenvs/default-3.8/bin/..//../lb/python38.zip',
    '/home/josh/.virtualenvs/default-3.8/bin/..//../lib/python3.8',
    '/home/josh/.virtualenvs/default-3.8/bin/..//../lib/python3.8/lib-dynload',
    '/home/josh/.virtualenvs/default-3.8/bin/..//../lib/python3.8/site-packages',
    '/home/josh/.virtualenvs/default-3.8/bin/..//lib64/python38.zip',
    '/home/josh/.virtualenvs/default-3.8/bin/..//lib64/python3.8',
    '/home/josh/.virtualenvs/default-3.8/bin/..//lib64/python3.8/lib-dynload',
    '/home/josh/.virtualenvs/default-3.8/bin/..//lib64/python3.8/site-packages',
    '/home/josh/.virtualenvs/default-3.8/bin/..//../lib64/python38.zip',
    '/home/josh/.virtualenvs/default-3.8/bin/..//../lib64/python3.8',
    '/home/josh/.virtualenvs/default-3.8/bin/..//../lib64/python3.8/lib-dynload',
    '/home/josh/.virtualenvs/default-3.8/bin/..//../lib64/python3.8/site-packages',
    '',
  ]
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized

Manually setting PYTHONPATH does not appear to have an effect

joshbode commented 4 years ago 
$ jupyter console --kernel=python3
>>> import sys
>>> print("\n".join(sorted(sys.path)))

/home/josh
/home/josh/.ipython
/home/josh/.virtualenvs/default-3.8/lib/python3.8/site-packages
/home/josh/.virtualenvs/default-3.8/lib/python3.8/site-packages/IPython/extensions
/usr/lib/python3.8
/usr/lib/python3.8/lib-dynload
/usr/lib/python38.zip

vs:

$ jupyter console --kernel=xpython
>>> import sys
>>> print("\n".join(sorted(sys.path)))
.
/home/josh
/home/josh/.virtualenvs/default-3.8/lib/python3.8
/home/josh/.virtualenvs/default-3.8/lib/python3.8/lib-dynload
/home/josh/.virtualenvs/default-3.8/lib/python3.8/site-packages
/home/josh/.virtualenvs/default-3.8/lib/python38.zip
/home/josh/.virtualenvs/default-3.8/lib64/python3.8
/home/josh/.virtualenvs/default-3.8/lib64/python3.8/lib-dynload
/home/josh/.virtualenvs/default-3.8/lib64/python3.8/site-packages
/home/josh/.virtualenvs/default-3.8/lib64/python38.zip
/home/josh/.virtualenvs/lb/python38.zip      <------------------- typo?
/home/josh/.virtualenvs/lib/python3.8
/home/josh/.virtualenvs/lib/python3.8/lib-dynload
/home/josh/.virtualenvs/lib/python3.8/site-packages
/home/josh/.virtualenvs/lib64/python3.8
/home/josh/.virtualenvs/lib64/python3.8/lib-dynload
/home/josh/.virtualenvs/lib64/python3.8/site-packages
/home/josh/.virtualenvs/lib64/python38.zip

possible typo, maybe unrelated: https://github.com/jupyter-xeus/xeus-python-wheel/blob/7dc1366935dae3463b5575d0f327a91a9cf96469/CMakeLists.txt#L196