Errors when building/installing ssh==1.7.14

[nvie]

Installing ssh==1.7.14 leads to this error, when installed with pip-accel:

(pip-accel) Building binary distribution of ssh (1.7.14) ..
running bdist_dumb
running build
running build_py
installing to build/bdist.macosx-10.8-intel/dumb
running install
running install_lib
running install_egg_info
running egg_info
writing requirements to ssh.egg-info/requires.txt
writing ssh.egg-info/PKG-INFO
writing top-level names to ssh.egg-info/top_level.txt
writing dependency_links to ssh.egg-info/dependency_links.txt
reading manifest file 'ssh.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'ssh.egg-info/SOURCES.txt'
removing 'build/bdist.macosx-10.8-intel/dumb/Users/nvie/.virtualenvs/foo/lib/python2.7/site-packages/ssh-1.7.14-py2.7.egg-info' (and everything under it)
Copying ssh.egg-info to build/bdist.macosx-10.8-intel/dumb/Users/nvie/.virtualenvs/foo/lib/python2.7/site-packages/ssh-1.7.14-py2.7.egg-info
running install_scripts
Traceback (most recent call last):
  File "setup.py", line 67, in <module>
    **kw
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/core.py", line 152, in setup
    dist.run_commands()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/command/bdist_dumb.py", line 123, in run
    owner=self.owner, group=self.group)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/cmd.py", line 392, in make_archive
    owner=owner, group=group)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/archive_util.py", line 237, in make_archive
    filename = func(base_name, base_dir, **kwargs)
TypeError: make_tarball() got an unexpected keyword argument 'owner'
(pip-accel) Failed to build binary distribution!

Installing with pip works just fine. Any clues?

[xolox]

Thanks for the feedback! The stack trace kind of looks like a bug in distutils unless I'm misunderstanding things completely... Basically pip-accel executes python setup.py bdist_dumb and expects it to work (which it did on the 50+ modules I tried it on). I'll see if I can reproduce this on my Mac this weekend.

[xolox]

I'm running Mac OS X Lion 10.7.5 and the command pip-accel install ssh==1.7.14 works for me. Looking at the output you posted you seem to be running Mac OS X 10.8. Yes, that's it, I'm able to reproduce it on Mac OS X 10.8 (tested just now). I suppose it's to do with the version of distutils included in 10.8. I don't think this is a bug in pip-accel but I can see if there's a solution.

[xolox]

I can reproduce it on my girlfriend's MacBook running Mac OS X 10.8, but the system wide *.py files referenced in the stack trace don't exist, there's just *.pyc and *.pyo files, so I can't even take a look at the source code (I don't like that Mac OS X bundles Python like this, but I digress).

However fortunately I did take a look at the setup.py and setup_helper.py scripts in the ssh distribution and I think the problem is caused there. The setup.py script for version 1.7.14 contains this bit of code:

if sys.platform == 'darwin':
    import setup_helper
    setup_helper.install_custom_make_tarball()

Inside setup_helper.py I found this:

_custom_formats = {
    'gztar': (make_tarball, [('compress', 'gzip')], "gzip'ed tar-file"),
    'bztar': (make_tarball, [('compress', 'bzip2')], "bzip2'ed tar-file"),
    'ztar':  (make_tarball, [('compress', 'compress')], "compressed tar file"),
    'tar':   (make_tarball, [('compress', None)], "uncompressed tar file"),
}

# Hack in and insert ourselves into the distutils code base
def install_custom_make_tarball():
    distutils.archive_util.ARCHIVE_FORMATS.update(_custom_formats)

As the comment points out this is a monkey patch of distutils, and I'm convinced this is the cause of the problem. I found the ssh project on GitHub to report the bug, however I noticed that the readme says that ssh was a fork of Paramiko and has since been merged back, making the project / repository defunct. So I guess there's no point in reporting the bug. All I can suggest to you is to try a different version of ssh or maybe Paramiko?

[nvie]

Wow, that's some awesome support, Peter. Cool. I believe ssh is s secondary dependency—I don't use it directly. I'll find out which project uses it and see if they can upgrade to paramiko anyway. Thanks for the great reply here.

[mbrochh]

I fell into the same trap and could get around it by pinning to paramiko==dev in my requirements.txt

[l0kix2]

So latest paramiko have this annoying hack https://github.com/paramiko/paramiko/blob/master/setup.py#L52

Pip installs it without complains, so maybe pip-accel somehow should do it? This issue closed, but it can't see workaround or smth.

[xolox]

@l0kix2: Just got an email notification from your message here. I don't have time to look into this right now but I will definitely forget if I don't follow up on this in some way, so I'll reopen this issue as a reminder to investigate. If things stay quiet for too long then feel free to bump this issue to remind me ;-)

[l0kix2]

Thanks, i really don't know how to handle it for a long time. When i'm trying to install fabric on mac os via pip-accel it installs paramiko, which fails with TypeError: make_tarball() got an unexpected keyword argument 'owner'

[enkeboll]

Bump, happening over here as well with paramiko-1.12.4 on OS X 10.10.5 with Python 2.7.10 as the base.

[xolox]

Hi @l0kix2 and @enkeboll,

As I explained before in this issue the bug here is not in pip-accel but in the SSH and Paramiko packages: Various versions of both packages (including apparently the master branch of Paramiko :-s) include a Mac OS X specific customization of the python setup.py bdist_dumb command as implemented by distutils. This customization has become incompatible with newer versions of distutils which is why installation of SSH and Paramiko via pip-accel on Mac OS X is now broken.

However I understand that just pointing out a bug elsewhere doesn't really help you at all :-). Given that I understand what the problem is here and really it shouldn't be that hard to fix it I decided to create a pull request on the Paramiko project to get this fixed, you can find the pull request here: paramiko/paramiko#582.

In the short term, if the version of Paramiko on the master branch satisfies your requirements you can install it as follows:

pip-accel install https://github.com/xolox/paramiko/archive/fix-bdist-dumb-mac-os-x.zip

Let's hope my pull request is merged quickly :-).

[enkeboll]

Thanks a lot for the update;.

[xolox]

My pull request was just merged which means the Paramiko master branch is now installable via pip-accel on Mac OS X. Until a proper release is made you should be able to use e.g. this syntax:

pip-accel install https://github.com/paramiko/paramiko/archive/master.zip

I believe this is the best I can do so I'm going to go ahead and close this issue now. Thanks for the feedback everyone!