0.20.0 issues with Python 3.6 and 3.7 on MacOS

[carlmontanari]

Bug report

0.20.0 on MacOS for Python versions 3.6 and 3.7 installs improperly. Python 3.8 on MacOS works as expected. I have tested this locally on my Mac and an Ubuntu 20 host, as well as on macOS-lastest and ubuntu-latest via GitHub Actions. Ubuntu works on all versions as expected.

Steps to reproduce:

1. Example code that produces error.

python -c 'from ssh2.session import Session' or importing from ssh2.channel import Channel for example.

2. Stack trace or error messages.

See below.

Expected behaviour: Import is successful

Actual behaviour: Python 3.6 and 3.7 raise ModuleNotFoundError

Additional info:

Installing via pip or from source both fails for 3.6 and 3.7, pip installing 3.8 works as expected.

Fresh venv ++ upgrade pip ++ install ssh2-python on 3.6.8:

 carl@impostore  ~/Desktop  python3.6 --version   
Python 3.6.8
 carl@impostore  ~/Desktop  python3.6 -m venv venv
 carl@impostore  ~/Desktop  source venv/bin/activate
(venv)  carl@impostore  ~/Desktop  python --version
Python 3.6.8
(venv)  carl@impostore  ~/Desktop  pip --version
pip 18.1 from /Users/carl/Desktop/venv/lib/python3.6/site-packages/pip (python 3.6)
(venv)  carl@impostore  ~/Desktop  pip install -U pip
Collecting pip
  Using cached https://files.pythonhosted.org/packages/4e/5f/528232275f6509b1fff703c9280e58951a81abe24640905de621c9f81839/pip-20.2.3-py2.py3-none-any.whl
Installing collected packages: pip
  Found existing installation: pip 18.1
    Uninstalling pip-18.1:
      Successfully uninstalled pip-18.1
Successfully installed pip-20.2.3
(venv)  carl@impostore  ~/Desktop  pip --version
pip 20.2.3 from /Users/carl/Desktop/venv/lib/python3.6/site-packages/pip (python 3.6)
(venv)  carl@impostore  ~/Desktop  pip install ssh2-python
Processing /Users/carl/Library/Caches/pip/wheels/b2/3a/d2/3bf68b0c0ad193c8b07345a6dd9e9ee7376b175c2cbc1de42d/ssh2_python-0.20.0-cp36-cp36m-macosx_10_9_x86_64.whl
Installing collected packages: ssh2-python
Successfully installed ssh2-python-0.20.0
(venv)  carl@impostore  ~/Desktop  python -c 'from ssh2.session import Session'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'ssh2.session'
(venv)  ✘  carl@impostore  ~/Desktop  echo $?
1

Same thing on 3.7.7:

 carl@impostore  ~/Desktop  python3.7 --version
Python 3.7.7
 carl@impostore  ~/Desktop  python3.7 -m venv venv
 carl@impostore  ~/Desktop  source venv/bin/activate
(venv)  carl@impostore  ~/Desktop  python --version
Python 3.7.7
(venv)  carl@impostore  ~/Desktop  pip --version
pip 19.2.3 from /Users/carl/Desktop/venv/lib/python3.7/site-packages/pip (python 3.7)
(venv)  carl@impostore  ~/Desktop  pip install -U pip
Collecting pip
  Using cached https://files.pythonhosted.org/packages/4e/5f/528232275f6509b1fff703c9280e58951a81abe24640905de621c9f81839/pip-20.2.3-py2.py3-none-any.whl
Installing collected packages: pip
  Found existing installation: pip 19.2.3
    Uninstalling pip-19.2.3:
      Successfully uninstalled pip-19.2.3
Successfully installed pip-20.2.3
(venv)  carl@impostore  ~/Desktop  pip install ssh2-python
Collecting ssh2-python
  Using cached ssh2-python-0.20.0.tar.gz (535 kB)
Using legacy 'setup.py install' for ssh2-python, since package 'wheel' is not installed.
Installing collected packages: ssh2-python
    Running setup.py install for ssh2-python ... |python -c 'from ssh2.session import Sessiondone
Successfully installed ssh2-python-0.20.0
(venv)  carl@impostore  ~/Desktop  python -c 'from ssh2.session import Session'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'ssh2.session'
(venv)  ✘  carl@impostore  ~/Desktop  echo $?
1

And working w/ 3.8:

 carl@impostore  ~/Desktop  python --version
Python 3.8.5
 carl@impostore  ~/Desktop  sw_vers -productVersion
10.15.6
 carl@impostore  ~/Desktop  python -m venv venv
 carl@impostore  ~/Desktop  source venv/bin/activate
(venv)  carl@impostore  ~/Desktop  python --version
Python 3.8.5
(venv)  carl@impostore  ~/Desktop  pip --version
pip 20.1.1 from /Users/carl/Desktop/venv/lib/python3.8/site-packages/pip (python 3.8)
(venv)  carl@impostore  ~/Desktop  pip install -U pip
Collecting pip
  Using cached pip-20.2.3-py2.py3-none-any.whl (1.5 MB)
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 20.1.1
    Uninstalling pip-20.1.1:
      Successfully uninstalled pip-20.1.1
Successfully installed pip-20.2.3
(venv)  carl@impostore  ~/Desktop  pip --version     
pip 20.2.3 from /Users/carl/Desktop/venv/lib/python3.8/site-packages/pip (python 3.8)
(venv)  carl@impostore  ~/Desktop  pip install ssh2-python
Collecting ssh2-python
  Using cached ssh2_python-0.20.0-cp38-cp38-macosx_10_15_x86_64.whl (2.4 MB)
Installing collected packages: ssh2-python
Successfully installed ssh2-python-0.20.0
(venv)  carl@impostore  ~/Desktop  pip list | grep ssh2
ssh2-python 0.20.0
(venv)  carl@impostore  ~/Desktop  python -c 'from ssh2.session import Session'
(venv)  carl@impostore  ~/Desktop  echo $?
0
(venv)  carl@impostore  ~/Desktop  

The actual installed .pyx files look the same between versions, and the Channel class for example is definitely there :)

(venv)  carl@impostore  ~/Desktop  python -c "import ssh2; print(ssh2.__file__)"
/Users/carl/Desktop/venv/lib/python3.6/site-packages/ssh2/__init__.py
(venv)  carl@impostore  ~/Desktop  cat /Users/carl/Desktop/venv/lib/python3.6/site-packages/ssh2/channel.pyx| grep Channel
from exceptions import ChannelError
cdef object PyChannel(c_ssh2.LIBSSH2_CHANNEL *channel, Session session):
    cdef Channel _channel = Channel.__new__(Channel, session)
cdef class Channel:
        :raises: :py:class:`ssh2.exceptions.ChannelError` on errors executing
        :raises: :py:class:`ssh2.exceptions.ChannelError` on errors requesting

I'm a outa my depth on the Cython/bindings bits so not super sure where else to be poking about, but happy to check whatever you need me to look at!

Glad to see this project getting some commits lately, I'm a big fan :D!

Carl

[davnat]

I am also experiencing this. I have an Ubuntu 20.04 with python 3.8 and an Alpine based Docker image with python 3.7 for CI: it seems that on 3.7 most cpython shared libraries are missing.

On Ubuntu I have:

$ ls .venv/lib/python3.8/site-packages/ssh2
agent.c                                     exceptions.cpython-38-x86_64-linux-gnu.so  listener.pxd                              sftp_handle.c
agent.cpython-38-x86_64-linux-gnu.so        exceptions.pyx                             listener.pyx                              sftp_handle.cpython-38-x86_64-linux-gnu.so
agent.pxd                                   fileinfo.c                                 pkey.c                                    sftp_handle.pxd
agent.pyx                                   fileinfo.cpython-38-x86_64-linux-gnu.so    pkey.cpython-38-x86_64-linux-gnu.so       sftp_handle.pyx
channel.c                                   fileinfo.pxd                               pkey.pxd                                  sftp.pxd
channel.cpython-38-x86_64-linux-gnu.so      fileinfo.pyx                               pkey.pyx                                  sftp.pyx
channel.pxd                                 find_eol.c                                 publickey.c                               statinfo.c
channel.pyx                                 find_eol.cpython-38-x86_64-linux-gnu.so    publickey.cpython-38-x86_64-linux-gnu.so  statinfo.cpython-38-x86_64-linux-gnu.so
c_pkey.pxd                                  find_eol.h                                 publickey.pxd                             statinfo.pxd
c_sftp.pxd                                  __init__.pxd                               publickey.pyx                             statinfo.pyx
c_ssh2.pxd                                  __init__.py                                __pycache__                               utils.c
c_stat.pxd                                  knownhost.c                                session.c                                 utils.cpython-38-x86_64-linux-gnu.so
error_codes.c                               knownhost.cpython-38-x86_64-linux-gnu.so   session.cpython-38-x86_64-linux-gnu.so    utils.pxd
error_codes.cpython-38-x86_64-linux-gnu.so  knownhost.pxd                              session.pxd                               utils.pyx
error_codes.pxd                             knownhost.pyx                              session.pyx                               _version.py
error_codes.pyx                             listener.c                                 sftp.c
exceptions.c                                listener.cpython-38-x86_64-linux-gnu.so    sftp.cpython-38-x86_64-linux-gnu.so

while in the docker image:

$ ls /usr/local/lib/python3.7/site-packages/ssh2/
__init__.pxd             channel.pyx                               libssh2.so                   sftp.pxd
__init__.py              error_codes.pxd                           libssh2.so.1                 sftp.pyx
__pycache__              error_codes.pyx                           libssh2.so.1.0.1             sftp_handle.pxd
_version.py              exceptions.pyx                            listener.pxd                 sftp_handle.pyx
agent.pxd                fileinfo.pxd                              listener.pyx                 statinfo.pxd
agent.pyx                fileinfo.pyx                              pkey.pxd                     statinfo.pyx
c_pkey.pxd               find_eol.c                                pkey.pyx                     utils.pxd
c_sftp.pxd               find_eol.cpython-37m-x86_64-linux-gnu.so  publickey.pxd                utils.pyx
c_ssh2.pxd               find_eol.h                                publickey.pyx
c_stat.pxd               knownhost.pxd                             session.pxd
channel.pxd              knownhost.pyx                             session.pyx
$ python3
Python 3.7.9 (default, Aug 18 2020, 04:02:05) 
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from ssh2.session import Session
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'ssh2.session'

[pkittenis]

Thanks for the interest and report.

This is a bad source package on pypi that was uploaded without the .c files included. It seems that they are not included by default on systems with cython installed, have made changes so they are always included.

On older Linux distributions update pip to get binary wheels instead of building from source. pip install -U pip. See also documentation.

Resolved by 0.22.0