Can not run Crossbar on asyncio

WebDevelopNemanja commented 3 years ago

Hi, I have a problem with to start crossbar on asyncio

Here is simple app:


#!/usr/bin/env python3

import asyncio
import txaio

txaio.use_asyncio()

from autobahn.asyncio.wamp import ApplicationSession
from app.utilities.core.extra import *

class WSSession(ApplicationSession):

    def __init__(self, config=None):
        super(ApplicationSession, self).__init__(config)
        self.log = logger(log2file='ws-session')

    def __repr__(self):
        return '<WSSession>'

    def onConnect(self):
        self.log.info("{} component connected!".format(self.__repr__()))
        self.join(self.config.realm)

    def onLeave(self, details):
        self.log.info("{} component left!".format(self.__repr__()))
        self.disconnect()

    def onDisconnect(self):
        self.log.info("{} component disconnected!".format(self.__repr__()))
        try:
            loop = asyncio.get_event_loop()
            loop.stop()
        except:
            pass

    def onJoin(self, details):
        pass

and the exception stack:

2021-02-24T12:07:52+0100 [Controller  22778] Ok, Realm realm001 has started Role role001
2021-02-24T12:07:52+0100 [Router      22779] component loading failed
Traceback (most recent call last):
  File "/Users/d3v3l0p3r90/work/etaxiapp-backend/lib/python3.8/site-packages/autobahn/wamp/websocket.py", line 95, in onMessage
    self._session.onMessage(msg)
  File "/Users/d3v3l0p3r90/work/etaxiapp-backend/lib/python3.8/site-packages/autobahn/wamp/protocol.py", line 1025, in onMessage
    on_reply = txaio.as_future(endpoint.fn, *invoke_args, **invoke_kwargs)
  File "/Users/d3v3l0p3r90/work/etaxiapp-backend/lib/python3.8/site-packages/txaio/tx.py", line 366, in as_future
    return maybeDeferred(fun, *args, **kwargs)
  File "/Users/d3v3l0p3r90/work/etaxiapp-backend/lib/python3.8/site-packages/twisted/internet/defer.py", line 151, in maybeDeferred
    result = f(*args, **kw)
--- <exception caught here> ---
  File "/Users/d3v3l0p3r90/work/etaxiapp-backend/lib/python3.8/site-packages/crossbar/worker/router.py", line 712, in start_router_component
    create_component = _appsession_loader(config)
  File "/Users/d3v3l0p3r90/work/etaxiapp-backend/lib/python3.8/site-packages/crossbar/worker/__init__.py", line 78, in _appsession_loader
    raise ApplicationError(
autobahn.wamp.exception.ApplicationError: ApplicationError(error=<crossbar.error.class_import_failed>, args=['Failed to import class \'run.WSSession\'\nTraceback (most recent call last):\n  File "/Users/d3v3l0p3r90/work/etaxiapp-backend/lib/python3.8/site-packages/autobahn/wamp/protocol.py", line 1025, in onMessage\n    on_reply = txaio.as_future(endpoint.fn, *invoke_args, **invoke_kwargs)\n  File "/Users/d3v3l0p3r90/work/etaxiapp-backend/lib/python3.8/site-packages/txaio/tx.py", line 366, in as_future\n    return maybeDeferred(fun, *args, **kwargs)\n  File "/Users/d3v3l0p3r90/work/etaxiapp-backend/lib/python3.8/site-packages/twisted/internet/defer.py", line 151, in maybeDeferred\n    result = f(*args, **kw)\n  File "/Users/d3v3l0p3r90/work/etaxiapp-backend/lib/python3.8/site-packages/crossbar/worker/router.py", line 712, in start_router_component\n    create_component = _appsession_loader(config)\n--- <exception caught here> ---\n  File "/Users/d3v3l0p3r90/work/etaxiapp-backend/lib/python3.8/site-packages/crossbar/worker/__init__.py", line 65, in _appsession_loader\n    module = importlib.import_module(module_name)\n  File "/Users/d3v3l0p3r90/.pyenv/versions/3.8.7/lib/python3.8/importlib/__init__.py", line 127, in import_module\n    return _bootstrap._gcd_import(name[level:], package, level)\n  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import\n    \n  File "<frozen importlib._bootstrap>", line 991, in _find_and_load\n    \n  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked\n    \n  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked\n    \n  File "<frozen importlib._bootstrap_external>", line 783, in exec_module\n    \n  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed\n    \n  File "/Users/d3v3l0p3r90/work/etaxiapp-backend/src/app/api/core/ws/run.py", line 5, in <module>\n    txaio.use_asyncio()\n  File "/Users/d3v3l0p3r90/work/etaxiapp-backend/lib/python3.8/site-packages/txaio/__init__.py", line 122, in use_asyncio\n    raise RuntimeError("Explicitly using \'{}\' already".format(_explicit_framework))\nbuiltins.RuntimeError: Explicitly using \'twisted\' already\n'], kwargs={'pythonpath': ['/Users/d3v3l0p3r90/work/etaxiapp-backend/src/app/api/core/ws', '/Users/d3v3l0p3r90/work/etaxiapp-backend/src/app/api/core/ws', '/Users/d3v3l0p3r90/work/etaxiapp-backend/src/app/api', '/Users/d3v3l0p3r90/work/etaxiapp-backend/bin', '/Users/d3v3l0p3r90/.pyenv/versions/3.8.7/lib/python38.zip', '/Users/d3v3l0p3r90/.pyenv/versions/3.8.7/lib/python3.8', '/Users/d3v3l0p3r90/.pyenv/versions/3.8.7/lib/python3.8/lib-dynload', '/Users/d3v3l0p3r90/work/etaxiapp-backend/lib/python3.8/site-packages', '/Users/d3v3l0p3r90/work/etaxiapp-backend/src']}, enc_algo=None, callee=None, callee_authid=None, callee_authrole=None, forward_for=None)

2021-02-24T12:07:52+0100 [Controller  22778] Could not start node: Traceback (most recent call last):
--- <exception caught here> ---
  File "/Users/d3v3l0p3r90/work/etaxiapp-backend/lib/python3.8/site-packages/crossbar/node/node.py", line 468, in start
    res = yield self.personality.Node.boot(self)
  File "/Users/d3v3l0p3r90/work/etaxiapp-backend/lib/python3.8/site-packages/crossbar/node/node.py", line 608, in boot_from_config
    yield d
  File "/Users/d3v3l0p3r90/work/etaxiapp-backend/lib/python3.8/site-packages/crossbar/node/node.py", line 581, in configure_worker
    yield config_fn(worker_logname, worker_id, worker)
  File "/Users/d3v3l0p3r90/work/etaxiapp-backend/lib/python3.8/site-packages/crossbar/node/node.py", line 702, in _configure_native_worker_router
    yield self._controller.call('crossbar.worker.{}.start_router_component'.format(worker_id), component_id, component, options=CallOptions())
autobahn.wamp.exception.ApplicationError: ApplicationError(error=<crossbar.error.class_import_failed>, args=['Failed to import class \'run.WSSession\'\nTraceback (most recent call last):\n  File "/Users/d3v3l0p3r90/work/etaxiapp-backend/lib/python3.8/site-packages/autobahn/wamp/protocol.py", line 1025, in onMessage\n    on_reply = txaio.as_future(endpoint.fn, *invoke_args, **invoke_kwargs)\n  File "/Users/d3v3l0p3r90/work/etaxiapp-backend/lib/python3.8/site-packages/txaio/tx.py", line 366, in as_future\n    return maybeDeferred(fun, *args, **kwargs)\n  File "/Users/d3v3l0p3r90/work/etaxiapp-backend/lib/python3.8/site-packages/twisted/internet/defer.py", line 151, in maybeDeferred\n    result = f(*args, **kw)\n  File "/Users/d3v3l0p3r90/work/etaxiapp-backend/lib/python3.8/site-packages/crossbar/worker/router.py", line 712, in start_router_component\n    create_component = _appsession_loader(config)\n--- <exception caught here> ---\n  File "/Users/d3v3l0p3r90/work/etaxiapp-backend/lib/python3.8/site-packages/crossbar/worker/__init__.py", line 65, in _appsession_loader\n    module = importlib.import_module(module_name)\n  File "/Users/d3v3l0p3r90/.pyenv/versions/3.8.7/lib/python3.8/importlib/__init__.py", line 127, in import_module\n    return _bootstrap._gcd_import(name[level:], package, level)\n  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import\n    \n  File "<frozen importlib._bootstrap>", line 991, in _find_and_load\n    \n  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked\n    \n  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked\n    \n  File "<frozen importlib._bootstrap_external>", line 783, in exec_module\n    \n  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed\n    \n  File "/Users/d3v3l0p3r90/work/etaxiapp-backend/src/app/api/core/ws/run.py", line 5, in <module>\n    txaio.use_asyncio()\n  File "/Users/d3v3l0p3r90/work/etaxiapp-backend/lib/python3.8/site-packages/txaio/__init__.py", line 122, in use_asyncio\n    raise RuntimeError("Explicitly using \'{}\' already".format(_explicit_framework))\nbuiltins.RuntimeError: Explicitly using \'twisted\' already\n'], kwargs={'pythonpath': ['/Users/d3v3l0p3r90/work/etaxiapp-backend/src/app/api/core/ws', '/Users/d3v3l0p3r90/work/etaxiapp-backend/src/app/api/core/ws', '/Users/d3v3l0p3r90/work/etaxiapp-backend/src/app/api', '/Users/d3v3l0p3r90/work/etaxiapp-backend/bin', '/Users/d3v3l0p3r90/.pyenv/versions/3.8.7/lib/python38.zip', '/Users/d3v3l0p3r90/.pyenv/versions/3.8.7/lib/python3.8', '/Users/d3v3l0p3r90/.pyenv/versions/3.8.7/lib/python3.8/lib-dynload', '/Users/d3v3l0p3r90/work/etaxiapp-backend/lib/python3.8/site-packages', '/Users/d3v3l0p3r90/work/etaxiapp-backend/src']}, enc_algo=None, callee=None, callee_authid=None, callee_authrole=None, forward_for=None)

2021-02-24T12:07:52+0100 [Controller  22778] sending TERM to subprocess 22779

it saids on end that: Explicitly using 'twisted' already

Thank you

oberstet commented 3 years ago

yes, router and container components must be written using twisted. if you have to use asyncio, you must run it as a guest worker or completely external to crossbar

aturkenovorion commented 2 years ago

yes, router and container components must be written using twisted. if you have to use asyncio, you must run it as a guest worker or completely external to crossbar

Hello, @oberstet Im developing micro framework for WAMP. And i faced with problem Explicitly using 'twisted' already. Are you going to develop asyncio implementation for crossbar worker? Or I can write an ApplicationSession only on twisted, which will execute async coroutines. Can you give some advices? Thanks for answering

oberstet commented 2 years ago

many options, it all depends;) here are some clues, hope that helps!

ApplicationSession/Twisted can be run hosted inside a Crossbar.io Container or Router Worker (and also Guest Worker)
ApplicationSession/asyncio can only be run as a Crossbar.io Guest Worker
we currently don't have plans to add ApplicationSession/asyncio based Crossbar.io Container (or Router) Worker types
co-routines can be used with both Twisted and asyncio
co-routines on Twisted (old-school style, works since Python 2): https://github.com/crossbario/autobahn-python/blob/master/examples/twisted/wamp/overview/backend.py
co-routines on asyncio (new-school Python 3 aio style): https://github.com/crossbario/autobahn-python/blob/master/examples/asyncio/wamp/overview/backend.py
nowerdays, you can also use new-school await/async style on Twisted .. can't find an example right now
you can also write code without co-routines, in a way, so that the code is source compatible across Twisted and asyncio https://github.com/crossbario/txaio#how-it-works

if you want your framework to be usable in the broadest range of situations, the txaio bare bones style is the way to go (this is what Autobahn is using internally)

if you want the maximum flexibiltiy/support in Crossbar.io, use ApplicationSession/Twisted, and either old- or new-school co-routines

aturkenovorion commented 2 years ago

Hello! @oberstet I used new-school co-routines for Twisted and it works fine! I wanna get my new overloaded autobahn.twisted.wamp.ApplicationSession from initialized object attribute (initialized_framework_name.session_factory: autobahn.twisted.wamp.OverLoadedApplicationSession). I tried to use type = 'function' instead of type = 'class' for crossbar worker configuration, but it does not work

oberstet commented 2 years ago

great! good to hear you've got sth working to start=) here are some more hints, maybe that helps:

so if you have a new class XYZ that is derived of autobahn.twisted.wamp.ApplicationSession, and if that class can be resolved and loaded from the Python that you use to start Crossbar.io, you then can have a Router/Container worker with a config that points to that class, eg here are examples:

find ~/scm/crossbario/crossbar-examples/ -name "*.json" -exec grep -Hi '"type": "class"' {} \;

a run-time object will be created from that class by Crossbar.io code - you can only specify the class

aturkenovorion commented 2 years ago

@oberstet , i found it: https://github.com/crossbario/crossbar/blob/4736e67b75bcd3f41be82e86008d9822299809ea/crossbar/worker/__init__.py#L26

what do you think if the loader could traverse through objects?

Example: timport_object('module.submodule.submodule.submodule.object.object.object.get_session_factory') -> autobahn.twisted.wamp.ApplicationSession

oberstet commented 2 years ago

no, this wouldn't work and this isn't required. anyways, sorry, I don't have time for this ..

crossbario / crossbar

Can not run Crossbar on asyncio #1858