LinkedInAttic / Zopkio

A Functional and Performance Test Framework for Distributed Systems
Apache License 2.0
159 stars 33 forks source link

Make zopkio's SSH work better out of the box #40

Open criccomini opened 9 years ago

criccomini commented 9 years ago

Out of the box, when running Zopkio, we get:

setup_suite() failed. See below for the trace.
Traceback (most recent call last):
  File "/tmp/samza-test/samza-integration-tests/lib/python2.7/site-packages/zopkio/test_runner.py", line 106, in run
    self.deployment_module.setup_suite()
  File "/tmp/samza-test/scripts/deployment.py", line 76, in setup_suite
    'hostname': host
  File "/tmp/samza-test/samza-integration-tests/lib/python2.7/site-packages/zopkio/deployer.py", line 76, in deploy
    self.install(unique_id, configs)
  File "/tmp/samza-test/samza-integration-tests/lib/python2.7/site-packages/zopkio/adhoc_deployer.py", line 99, in install
    with get_ssh_client(hostname) as ssh:
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/tmp/samza-test/samza-integration-tests/lib/python2.7/site-packages/zopkio/remote_host_helper.py", line 180, in get_ssh_client
    ssh.connect(hostname)
  File "/tmp/samza-test/samza-integration-tests/lib/python2.7/site-packages/paramiko/client.py", line 307, in connect
    look_for_keys, gss_auth, gss_kex, gss_deleg_creds, gss_host)
  File "/tmp/samza-test/samza-integration-tests/lib/python2.7/site-packages/paramiko/client.py", line 519, in _auth
    raise saved_exception
PasswordRequiredException: Private key file is encrypted

To fix this, the user has to manually add their public key to .ssh/authorized_keys on all hosts, and SSH to the boxs to ensure that the keyring password is cached (at least on OSX).

It'd be nice if:

  1. The password that Zopkio prompts for gets passed into the ssh.connect in remote_host_helper.py. Might also be nice if username were accepted as a param as well.
  2. Some docs were written to describe how to handle this issue (either with authorized_keys, or via the prompt).
  3. Maybe there's some better way of doing this? In the case where Zopkio is actually deploying to remote hosts, I can see where it'd b necessary to add your public key to authorized_hosts, but for localhost deployments, it seems a bit excessive.

Related: http://stackoverflow.com/questions/15579117/paramiko-using-encrypted-private-key-file-on-os-x

criccomini commented 9 years ago

Also: http://paramiko-docs.readthedocs.org/en/latest/api/client.html#paramiko.client.SSHClient

criccomini commented 9 years ago

On Linux, I can't get the SSH fix to work. I get:

Traceback (most recent call last):
  File "/tmp/samza-tests/samza-integration-tests/lib/python2.6/site-packages/zopkio/test_runner.py", line 106, in run
    self.deployment_module.setup_suite()
  File "/tmp/samza-tests/scripts/deployment.py", line 76, in setup_suite
    'hostname': host
  File "/tmp/samza-tests/samza-integration-tests/lib/python2.6/site-packages/zopkio/deployer.py", line 76, in deploy
    self.install(unique_id, configs)
  File "/tmp/samza-tests/samza-integration-tests/lib/python2.6/site-packages/zopkio/adhoc_deployer.py", line 99, in install
    with get_ssh_client(hostname) as ssh:
  File "/usr/lib64/python2.6/contextlib.py", line 16, in __enter__
    return self.gen.next()
  File "/tmp/samza-tests/samza-integration-tests/lib/python2.6/site-packages/zopkio/remote_host_helper.py", line 180, in get_ssh_client
    ssh.connect(hostname)
  File "/tmp/samza-tests/samza-integration-tests/lib/python2.6/site-packages/paramiko/client.py", line 307, in connect
    look_for_keys, gss_auth, gss_kex, gss_deleg_creds, gss_host)
  File "/tmp/samza-tests/samza-integration-tests/lib/python2.6/site-packages/paramiko/client.py", line 520, in _auth
    raise SSHException('No authentication methods available')
SSHException: No authentication methods available

I tried adding to authorized_keys, I tried SSH'ing to localhost, I tried entering both my SSH key password, and my SSH password to Zopkio. Nothing seems to work.

criccomini commented 9 years ago

This looks useful. http://stackoverflow.com/questions/14326665/ssh-python-with-paramiko-issue

pkilner commented 8 years ago

Yeah, I just built my own ssh and ftp libs. I will create a lib for running processes on clients next. Let me know if you would like some code.

Zopkio could use some Pep8 linting too!