Zerodeploy doesn't accept ParamikoMachine

[wormyourhonor]

from plumbum.machines.paramiko_machine import ParamikoMachine
from paramiko.client import MissingHostKeyPolicy
from plumbum import SshMachine
from rpyc.utils.zerodeploy import DeployedServer

# works fine
remoteStandard = SshMachine('127.0.0.1')
serverStandard = DeployedServer(remoteStandard)

# will raise exception
class ignoreIt(MissingHostKeyPolicy):
    def missing_host_key(self, client, hostname, key):
        pass

remoteParamiko = ParamikoMachine('127.0.0.1', missing_host_policy=ignoreIt())
serverParamiko = DeployedServer(remoteParamiko)

Receive traceback:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/user/py-rpyc/local/lib/python2.7/site-packages/rpyc-3.3.0-py2.7.egg/rpyc/utils/zerodeploy.py", line 86, in __init__
    self.proc = remote_machine.python.popen(script, new_session = True)
  File "/home/user/py-rpyc/local/lib/python2.7/site-packages/plumbum/machines/remote.py", line 93, in popen
    return self.remote.popen(self[args], **kwargs)
TypeError: popen() got an unexpected keyword argument 'new_session'

Note the difference in popen method definition for SshMachine vs ParamikoMachine

[tomerfiliba]

Hrrr. ParamikoMachine.popen should probably just ignore the new_session argument (as it runs in the same process). I'll get to it.

[tomerfiliba]

@wormyourhonor - I pushed a fix to Plumbum. Could you verify it works now? Thanks

[wormyourhonor]

hmmm....errors in zerodeploy script init now.

from plumbum.machines.paramiko_machine import ParamikoMachine
from paramiko.client import MissingHostKeyPolicy
from rpyc.utils.zerodeploy import DeployedServer

class ignoreIt(MissingHostKeyPolicy):
    def missing_host_key(self, client, hostname, key):
        pass

# works fine. woo-hoo!
remoteParamiko = ParamikoMachine('127.0.0.1', missing_host_policy=ignoreIt())

# will raise exception d'oh!
serverParamiko = DeployedServer(remoteParamiko)

here's the traceback...

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/user/py-bugfixes/local/lib/python2.7/site-packages/rpyc-3.3.0-py2.7.egg/rpyc/utils/zerodeploy.py", line 98, in __init__
    raise ProcessExecutionError(self.proc.argv, self.proc.returncode, line + stdout, stderr)
plumbum.commands.processes.ProcessExecutionError: Command line: ['cd', '/home/user', '&&', '/usr/bin/python', '/tmp/tmp.aj1f8mRulz/deployed-rpyc.py']
Exit code: 1
Stderr:  | Traceback (most recent call last):
         |   File "/tmp/tmp.aj1f8mRulz/deployed-rpyc.py", line 16, in <module>
         |     from rpyc.utils.server import ThreadedServer as ServerCls
         | ImportError: No module named rpyc.utils.server

Please note, I'm installing all tools (rpyc, plumbum, paramiko, & pycrypto) in a virtualenv. So, connecting to localhost should be like a generic remote python environment.

[wormyourhonor]

I see this issue is closed this issue after commit 02b9f35335b1c85cf27242cf7eda229c26c6fd0f

However, after reinstalling latest master of rpyc & plumbum then retrying the above use case, it still raises the same exception trying to launch zerodeploy.

[wormyourhonor]

Worth noting, just before attempting to run the remote deployed-rpyc.py script, the utils have yet to be copied.

E.g. when using ParamikoMachine the remote TMP directory looks like:

./tmp.Bu30bdt9gH/utils/zerodeploy.pyc
./tmp.Bu30bdt9gH/utils/zerodeploy.py
./tmp.Bu30bdt9gH/deployed-rpyc.py

However, SshMachine has:

./tmp.w2Hiap2w2z/deployed-rpyc.py
./tmp.w2Hiap2w2z/rpyc
./tmp.w2Hiap2w2z/rpyc/utils
./tmp.w2Hiap2w2z/rpyc/utils/__init__.py
./tmp.w2Hiap2w2z/rpyc/utils/zerodeploy.pyc
./tmp.w2Hiap2w2z/rpyc/utils/registry.pyc
./tmp.w2Hiap2w2z/rpyc/utils/factory.py
./tmp.w2Hiap2w2z/rpyc/utils/factory.pyc
./tmp.w2Hiap2w2z/rpyc/utils/classic.pyc
./tmp.w2Hiap2w2z/rpyc/utils/splitbrain.py
./tmp.w2Hiap2w2z/rpyc/utils/helpers.pyc
./tmp.w2Hiap2w2z/rpyc/utils/__init__.pyc
./tmp.w2Hiap2w2z/rpyc/utils/server.py
./tmp.w2Hiap2w2z/rpyc/utils/zerodeploy.py
./tmp.w2Hiap2w2z/rpyc/utils/classic.py
./tmp.w2Hiap2w2z/rpyc/utils/helpers.py
./tmp.w2Hiap2w2z/rpyc/utils/authenticators.py
./tmp.w2Hiap2w2z/rpyc/utils/registry.py
./tmp.w2Hiap2w2z/rpyc/lib
./tmp.w2Hiap2w2z/rpyc/lib/__init__.py
./tmp.w2Hiap2w2z/rpyc/lib/compat.pyc
./tmp.w2Hiap2w2z/rpyc/lib/colls.py
./tmp.w2Hiap2w2z/rpyc/lib/colls.pyc
./tmp.w2Hiap2w2z/rpyc/lib/__init__.pyc
./tmp.w2Hiap2w2z/rpyc/lib/compat.py
./tmp.w2Hiap2w2z/rpyc/__init__.py
./tmp.w2Hiap2w2z/rpyc/version.pyc
./tmp.w2Hiap2w2z/rpyc/core
./tmp.w2Hiap2w2z/rpyc/core/channel.pyc
./tmp.w2Hiap2w2z/rpyc/core/__init__.py
./tmp.w2Hiap2w2z/rpyc/core/consts.pyc
./tmp.w2Hiap2w2z/rpyc/core/vinegar.py
./tmp.w2Hiap2w2z/rpyc/core/netref.py
./tmp.w2Hiap2w2z/rpyc/core/protocol.pyc
./tmp.w2Hiap2w2z/rpyc/core/reactor.py
./tmp.w2Hiap2w2z/rpyc/core/async.pyc
./tmp.w2Hiap2w2z/rpyc/core/service.pyc
./tmp.w2Hiap2w2z/rpyc/core/brine.pyc
./tmp.w2Hiap2w2z/rpyc/core/service.py
./tmp.w2Hiap2w2z/rpyc/core/channel.py
./tmp.w2Hiap2w2z/rpyc/core/stream.py
./tmp.w2Hiap2w2z/rpyc/core/async.py
./tmp.w2Hiap2w2z/rpyc/core/consts.py
./tmp.w2Hiap2w2z/rpyc/core/stream.pyc
./tmp.w2Hiap2w2z/rpyc/core/netref.pyc
./tmp.w2Hiap2w2z/rpyc/core/__init__.pyc
./tmp.w2Hiap2w2z/rpyc/core/protocol.py
./tmp.w2Hiap2w2z/rpyc/core/vinegar.pyc
./tmp.w2Hiap2w2z/rpyc/core/brine.py
./tmp.w2Hiap2w2z/rpyc/__init__.pyc
./tmp.w2Hiap2w2z/rpyc/version.py

Looks like all the necessary libs aren't getting copied over in the case of ParamikoMachine.

[tomerfiliba]

please check again now

[wormyourhonor]

Checked against latest, 8cd9ebc63837b828bcb37e67129395c50794e6cf, and test case is PASS. Thanks for all the effort in getting this working.

One final note, once the script or python interpreter exits, there's a cleanup/deletion exception

Exception EOFError: EOFError() in <bound method DeployedServer.__del__ of <rpyc.utils.zerodeploy.DeployedServer object at 0x1fd9cd0>> ignored

If, however, I call DeployedServer.close or DeployedServer.__del__ manually, before exiting the python interpreter, all is good. I don't know if this is a concern or not.

UPDATE: Looks like the error occurs when DeployedServer.close is tries to close/exit _tmpdir_ctx.