Register new objects at an already running daemon

[pevogam]

Hi, what is the best way to register objects for remote access at an already running Pyro daemon and name server? I have the following code with the object of registering new remote modules and starting the two daemon loops if not already started:

import Pyro4

class DaemonLoop(threading.Thread):
    """Loop thread for sharing remote objects."""

    def __init__(self, pyro_daemon, ns_daemon=None):
        """
        Contruct the Pyro daemon thread.

        :param pyro_daemon: daemon for the remote python objects
        :type pyro_daemon: Pyro4.Daemon object
        :param ns_daemon: daemon for the name server
        :type ns_daemon: Pyro4.Daemon object
        """
        threading.Thread.__init__(self)
        self.pyro_daemon = pyro_daemon
        self.ns_daemon = ns_daemon
        self.daemon = True     # make this a Daemon Thread

    def run(self):
        """Run the Pyro4 daemon thread."""
        if self.ns_daemon is not None:
            self.pyro_daemon.combine(self.ns_daemon)
        self.pyro_daemon.requestLoop()

Pyro4.config.AUTOPROXY = False
Pyro4.config.REQUIRE_EXPOSE = False

# pyro daemon
try:
    pyro_daemon = Pyro4.Daemon(host=host, port=port)
    logging.debug("Pyro4 daemon started successfully")
    pyrod_running = False
# address already in use OS error
except OSError:
    pyro_daemon = Pyro4.Proxy("PYRO:" + Pyro4.constants.DAEMON_NAME + "@" + host + ":" + port)
    pyro_daemon.ping()
    logging.debug("Pyro4 daemon already started, available objects: %s", pyro_daemon.registered())
    pyrod_running = True

# namer server
try:
    ns_server = Pyro4.locateNS()
    logging.debug("Pyro4 name server already started")
    nsd_running = True
# failed to locate the nameserver error
except (OSError, Pyro4.errors.NamingError):
    from Pyro4 import naming
    ns_uri, ns_daemon, _bc_server = naming.startNS(host=host)
    ns_server = Pyro4.Proxy(ns_uri)
    logging.debug("Pyro4 name server started successfully with URI %s", ns_uri)
    nsd_running = False

# main retrieval of the local object
object_name = "os.path"
import importlib
local_object = importlib.import_module(object_name)

uri = pyro_daemon.register(local_object)
ns_server.register(object_name, uri)

loop = None
if not pyrod_running:
    loop = DaemonLoop(pyro_daemon, None if nsd_running else ns_daemon)
elif not nsd_running:
    loop = DaemonLoop(ns_daemon)

if loop is not None:
    loop.start()

However, calling ns_server.register via the proxy results in indefinite hanging during connection establishing. Am I missing something obvious? Is there a better and simpler way to achieve this?

This is the precise stack trace I am getting:

^CTraceback (most recent call last):
  File "<stdin>", line 368, in <module>
    ns_server.register(object_name, uri)
  File "/usr/lib/python3.6/site-packages/Pyro4/core.py", line 273, in __getattr__
    self._pyroGetMetadata()
  File "/usr/lib/python3.6/site-packages/Pyro4/core.py", line 600, in _pyroGetMetadata
    self.__pyroCreateConnection()
  File "/usr/lib/python3.6/site-packages/Pyro4/core.py", line 538, in __pyroCreateConnection
    msg = message.Message.recv(conn, [message.MSG_CONNECTOK, message.MSG_CONNECTFAIL], hmac_key=self._pyroHmacKey)
  File "/usr/lib/python3.6/site-packages/Pyro4/message.py", line 168, in recv
    msg = cls.from_header(connection.recv(cls.header_size))
  File "/usr/lib/python3.6/site-packages/Pyro4/socketutil.py", line 462, in recv
    return receiveData(self.sock, size)
  File "/usr/lib/python3.6/site-packages/Pyro4/socketutil.py", line 151, in receiveData
    data = sock.recv(size, socket.MSG_WAITALL)
KeyboardInterrupt

Thanks!

[irmen]

• your name server request loop isn't running yet at the time you're calling ns_server.register() right before where you create the DaemonLoop. So it's no surprise it doesn't react to the call.
• you should not be wanting to remotely register new objects into a running Pyro daemon. This imposes a huge security risk. Are you sure you want to do this?

[pevogam]

I first expected some symmetry between the pyro daemon and the name server in the request loop vs registration ordering, therefore my first guest was the one above, but I assume this difference (in comment lines below) is related to implementation details. For reference, rearranged and working version is:

import Pyro4

Pyro4.config.AUTOPROXY = False
Pyro4.config.REQUIRE_EXPOSE = False

# pyro daemon
try:
    pyro_daemon = Pyro4.Daemon(host=host)
    logging.debug("Pyro4 daemon started successfully")
    pyrod_running = False
# address already in use OS error
except OSError:
    pyro_daemon = Pyro4.Proxy("PYRO:" + Pyro4.constants.DAEMON_NAME + "@" + host)
    pyro_daemon.ping()
    logging.debug("Pyro4 daemon already started, available objects: %s",
                  pyro_daemon.registered())
    pyrod_running = True

# name server
try:
    ns_server = Pyro4.locateNS(host=host, port=port)
    logging.debug("Pyro4 name server already started")
    nsd_running = True
# network unreachable and failed to locate the nameserver error
except (OSError, Pyro4.errors.NamingError):
    from Pyro4 import naming
    ns_uri, ns_daemon, _bc_server = naming.startNS(host=host, port=port)
    ns_server = Pyro4.Proxy(ns_uri)
    logging.debug("Pyro4 name server started successfully with URI %s", ns_uri)
    nsd_running = False

# local_object = ...

# we should register to the pyro daemon before entering its loop
uri = pyro_daemon.register(local_object)
loop = None
if not pyrod_running:
    loop = DaemonLoop(pyro_daemon)
    loop.start()
elif not nsd_running:
    loop = DaemonLoop(ns_daemon)
    loop.start()
# we should register to the name server after entering its loop
ns_server.register(object_name, uri)

Regarding the second and more important point, the working version showed me that I could invoke multiple daemons each using different ports and registering objects at the name server so there is no need to access the same daemon. I hope the use case is clear though - sometimes it could be useful to dynamically generate remote objects (possibly at different times during a long process) that cannot all be registered and made available before the request loop or from a single deployed script. Of course, the security risks of rogue objects registered at a privileged daemon should be considered very carefully.

[irmen]

Thank you for following up and posting the solution to your issue, much appreciated.