SSH transport (new enhancement proposal)

[ThePrez]

With the release of the new itoolkit-utils package from IBM, it is now feasible to enable SSH as a transport mechanism for the various language toolkits, including Python.

The RPM, when installed, will put a binary, /QOpenSys/pkgs/bin/xmlservice-cgi that can be called through an SSH channel.

This enhancement is very important because it will enable a Python application from any platform (at least any platform where Paramiko works) to talk to IBM i and call RPG, Db2, etc. This can be done without the need to set up an Apache server on IBM i or an ODBC driver on the client. In that sense, it will be simpler than existing transports.

[jkyeung]

That is indeed a big deal, and a very welcome development.

Like most things related to this project, I would love to contribute, but I cannot make any commitments.

[ThePrez]

Pointer for whomever has a chance to look at this: to get paramiko working, you will likely have to install python3-cryptography, python3-pynacl, and python3-bcrypt via yum, then do your pip3 install paramiko

[ThePrez]

also yum group install "Developer Tools" (though it's a little overkill

[chrjorgensen]

Hello

I would love to have Paramiko working on i - I have a started using Storwize and FlashCopy technique and have to issue commands from IBM i to Storwize over a SSH connection. I know I could use Bash scripts for this, but I would prefer to use Python and Paramiko for a more interactive setup.

So I tried install Paramiko after having installed the following packages:

python3-bcrypt python3-cffi python3-cryptography libffi-devel openssl-devel python3-pynacl

But when I try to install Paramiko, it fails during build of the wheel...

The output is like this:

nccj@<ibmi>:/home/NCCJ/FlashCopy - pip install paramiko
Collecting paramiko
  Using cached https://files.pythonhosted.org/packages/4b/80/74dace9e48b0ef923633dfb5e48798f58a168e4734bca8ecfaf839ba051a/paramiko-2.6.0-py2.py3-none-any.whl
Collecting cryptography>=2.5 (from paramiko)
  Using cached https://files.pythonhosted.org/packages/c2/95/f43d02315f4ec074219c6e3124a87eba1d2d12196c2767fadfdc07a83884/cryptography-2.7.tar.gz
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... error
    ERROR: Complete output from command /QOpenSys/pkgs/bin/python3 /QOpenSys/pkgs/lib/python3.6/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmpaxytodif:
    ERROR: Traceback (most recent call last):
      File "/QOpenSys/pkgs/lib/python3.6/site-packages/pip/_vendor/pep517/_in_process.py", line 207, in <module>
        main()
      File "/QOpenSys/pkgs/lib/python3.6/site-packages/pip/_vendor/pep517/_in_process.py", line 197, in main
        json_out['return_val'] = hook(**hook_input['kwargs'])
      File "/QOpenSys/pkgs/lib/python3.6/site-packages/pip/_vendor/pep517/_in_process.py", line 69, in prepare_metadata_for_build_wheel
        return hook(metadata_directory, config_settings)
      File "/tmp/pip-build-env-nu2t9l3c/overlay/lib/python3.6/site-packages/setuptools/build_meta.py", line 155, in prepare_metadata_for_build_wheel
        self.run_setup()
      File "/tmp/pip-build-env-nu2t9l3c/overlay/lib/python3.6/site-packages/setuptools/build_meta.py", line 141, in run_setup        exec(compile(code, __file__, 'exec'), locals())
      File "setup.py", line 275, in <module>
        **keywords_with_side_effects(sys.argv)
      File "/tmp/pip-build-env-nu2t9l3c/overlay/lib/python3.6/site-packages/setuptools/__init__.py", line 145, in setup
        return distutils.core.setup(**attrs)
      File "/QOpenSys/pkgs/lib/python3.6/distutils/core.py", line 108, in setup
        _setup_distribution = dist = klass(attrs)
      File "/tmp/pip-build-env-nu2t9l3c/overlay/lib/python3.6/site-packages/setuptools/dist.py", line 444, in __init__
        k: v for k, v in attrs.items()
      File "/QOpenSys/pkgs/lib/python3.6/distutils/dist.py", line 281, in __init__
        self.finalize_options()
      File "/tmp/pip-build-env-nu2t9l3c/overlay/lib/python3.6/site-packages/setuptools/dist.py", line 732, in finalize_options
        ep.load()(self, ep.name, value)
      File "/tmp/pip-build-env-nu2t9l3c/overlay/lib/python3.6/site-packages/cffi/setuptools_ext.py", line 217, in cffi_modules
        add_cffi_module(dist, cffi_module)
      File "/tmp/pip-build-env-nu2t9l3c/overlay/lib/python3.6/site-packages/cffi/setuptools_ext.py", line 49, in add_cffi_module
        execfile(build_file_name, mod_vars)
      File "/tmp/pip-build-env-nu2t9l3c/overlay/lib/python3.6/site-packages/cffi/setuptools_ext.py", line 25, in execfile
        exec(code, glob, glob)
      File "src/_cffi_src/build_openssl.py", line 106, in <module>
        extra_link_args=extra_link_args(compiler_type()),
      File "src/_cffi_src/utils.py", line 57, in build_ffi_for_binding
        extra_link_args=extra_link_args,
      File "src/_cffi_src/utils.py", line 65, in build_ffi
        ffi = FFI()
      File "/tmp/pip-build-env-nu2t9l3c/overlay/lib/python3.6/site-packages/cffi/api.py", line 48, in __init__
        import _cffi_backend as backend
    ImportError: Could not load module /tmp/pip-build-env-nu2t9l3c/overlay/lib/python3.6/site-packages/_cffi_backend.so.
        The local-exec model was used for thread-local
           storage, but the module is not the main program.
    Examine the .loader section header with the 'dump -Hv' command.
    ----------------------------------------
ERROR: Command "/QOpenSys/pkgs/bin/python3 /QOpenSys/pkgs/lib/python3.6/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmpaxytodif" failed with error code 1 in /tmp/pip-install-wakg4tx7/cryptography

Seems like the cryptography python package on the installer is to old:

cryptography in /QOpenSys/pkgs/lib/python3.6/site-packages (2.2.2)

and Paramiko tries to fetch and build a newer version of cryptography:

**Collecting cryptography>=2.5 (from paramiko)**
  Using cached https://files.pythonhosted.org/packages/c2/95/f43d02315f4ec074219c6e3124a87eba1d2d12196c2767fadfdc07a83884/**cryptography-2.7**.tar.gz

Any ideas how to proceed?

[jkyeung]

For the time being, the simplest thing to do is probably install an older version of Paramiko, to work with the older version of cryptography. Maybe Paramiko 2.4.1 or 2.4.2. See this Stack Overflow question about installing older versions of things with pip.

[chrjorgensen]

@jkyeung Thanks, it worked!

nccj@<ibmi>:/home/NCCJ/FlashCopy - pip install paramiko==2.4.2
Collecting paramiko==2.4.2
  Downloading https://files.pythonhosted.org/packages/cf/ae/94e70d49044ccc234bfdba20114fa947d7ba6eb68a2e452d89b920e62227/paramiko-2.4.2-py2.py3-none-any.whl (193kB)
     |################################| 194kB 21.8MB/s
Requirement already satisfied: cryptography>=1.5 in /QOpenSys/pkgs/lib/python3.6/site-packages (from paramiko==2.4.2) (2.2.2)
Collecting pyasn1>=0.1.7 (from paramiko==2.4.2)
  Downloading https://files.pythonhosted.org/packages/7b/7c/c9386b82a25115cccf1903441bba3cbadcfae7b678a20167347fa8ded34c/pyasn1-0.4.5-py2.py3-none-any.whl (73kB)
     |################################| 81kB 1.2MB/s
Requirement already satisfied: bcrypt>=3.1.3 in /QOpenSys/pkgs/lib/python3.6/site-packages (from paramiko==2.4.2) (3.1.4)
Requirement already satisfied: pynacl>=1.0.1 in /QOpenSys/pkgs/lib/python3.6/site-packages (from paramiko==2.4.2) (1.2.1)
Requirement already satisfied: idna>=2.1 in /QOpenSys/pkgs/lib/python3.6/site-packages (from cryptography>=1.5->paramiko==2.4.2) (2.8)
Requirement already satisfied: asn1crypto>=0.21.0 in /QOpenSys/pkgs/lib/python3.6/site-packages (from cryptography>=1.5->paramiko==2.4.2) (0.24.0)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /QOpenSys/pkgs/lib/python3.6/site-packages (from cryptography>=1.5->paramiko==2.4.2) (1.11.5)
Requirement already satisfied: six>=1.4.1 in /QOpenSys/pkgs/lib/python3.6/site-packages (from cryptography>=1.5->paramiko==2.4.2) (1.12.0)
Requirement already satisfied: pycparser in /QOpenSys/pkgs/lib/python3.6/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.5->paramiko==2.4.2) (2.19)
Installing collected packages: pyasn1, paramiko
Successfully installed paramiko-2.4.2 pyasn1-0.4.5

[kadler]

This was delivered in #32, closing.

@chrjorgensen next time, please open a new issue. This would have been more appropriate on the general opensource issue tracker: https://bitbucket.org/ibmi/opensource/issues