search

crossbario / crossbar

Crossbar.io - WAMP application router
https://crossbar.io/
Other
2.05k stars 274 forks source link

use node controller centrally hosted security module #2032

Closed oberstet closed 2 years ago

oberstet commented 2 years ago 
 Error: 
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.7.10/x64/lib/python3.7/site-packages/twisted/internet/defer.py", line 1660, in _inlineCallbacks
    result = current_context.run(gen.send, result)
  File "/home/cbbuilder/actions-runner-crossbar/_work/crossbar/crossbar/crossbar/router/test/test_authorize.py", line 86, in test_authextra_wampcryptosign
    val = yield auth.hello("realm", details)
  File "/opt/hostedtoolcache/Python/3.7.10/x64/lib/python3.7/site-packages/twisted/internet/defer.py", line 1905, in unwindGenerator
    return _cancellableInlineCallbacks(gen)
  File "/opt/hostedtoolcache/Python/3.7.10/x64/lib/python3.7/site-packages/twisted/internet/defer.py", line 1815, in _cancellableInlineCallbacks
    _inlineCallbacks(None, gen, status)
  File "/opt/hostedtoolcache/Python/3.7.10/x64/lib/python3.7/site-packages/twisted/internet/defer.py", line 1663, in _inlineCallbacks
    status.deferred.callback(getattr(e, "value", None))
  File "/opt/hostedtoolcache/Python/3.7.10/x64/lib/python3.7/site-packages/twisted/internet/defer.py", line 660, in callback
    assert not isinstance(result, Deferred)
builtins.AssertionError: 
crossbar.router.test.test_authorize.TestDynamicAuth.test_authextra_wampcryptosign
oberstet commented 2 years ago

adjust _node_key in:

(cpy39_1) (base) oberstet@intel-nuci7:~/scm/crossbario/crossbar$ find crossbar -name "*.py" -exec grep -Hi "\._node_key" {} \;
crossbar/master/mrealm/controller.py:        self._node_key_hex = _read_node_key('.', private=True)['hex']
crossbar/master/mrealm/controller.py:        self._node_key = nacl.signing.SigningKey(self._node_key_hex, encoder=nacl.encoding.HexEncoder)
crossbar/master/node/controller.py:        self._node_key_hex = _read_node_key(cbdir, private=True)['hex']
crossbar/master/node/controller.py:        self._node_key = nacl.signing.SigningKey(self._node_key_hex, encoder=nacl.encoding.HexEncoder)
crossbar/master/node/controller.py:            node_key=self._node_key_hex)
crossbar/master/node/controller.py:                        mrealm_res['pubkey'] = self._node_key.verify_key.encode(encoder=nacl.encoding.RawEncoder)
crossbar/master/node/controller.py:            verify_key = self._node_key.verify_key.encode(encoder=nacl.encoding.RawEncoder)
crossbar/master/node/controller.py:            signed_msg = self._node_key.sign(raw_data)
oberstet commented 2 years ago

adjust sign_challenge in:

(cpy39_1) (base) oberstet@intel-nuci7:~/scm/crossbario/crossbar$ find . -name "*.py" -exec grep -Hi "sign_challenge" {} \;
./test/functests/helpers.py:        return self._key._sign_challenge(self, challenge)
./crossbar/shell/client.py:            sig = self._key._sign_challenge(self, challenge)
./crossbar/shell/client.py:        return self._key._sign_challenge(self, challenge)
./crossbar/network/test/test_api06_market.py:            signed_challenge = self._key.sign_challenge(self, challenge)
./crossbar/network/test/test_api09_catalog.py:            signed_challenge = self._key.sign_challenge(self, challenge)
./crossbar/network/test/test_api04_member.py:            signed_challenge = self._key.sign_challenge(self, challenge)
./crossbar/network/test/test_api08_catalog.py:            signed_challenge = self._key.sign_challenge(self, challenge)
./crossbar/network/test/test_api07_market.py:            signed_challenge = self._key.sign_challenge(self, challenge)
./crossbar/network/test/test_api03_logout.py:            signed_challenge = self._key.sign_challenge(self, challenge)
./crossbar/network/test/test_api02_onboard.py:            signed_challenge = self._key.sign_challenge(self, challenge)
./crossbar/network/test/test_api05_market.py:            signed_challenge = self._key.sign_challenge(self, challenge)
./crossbar/network/test/test_api01_echo.py:            signed_challenge = self._key.sign_challenge(self, challenge)
./crossbar/network/test/test_api10_api.py:            signed_challenge = self._key.sign_challenge(self, challenge)
./crossbar/network/test/test_api03_login.py:            signed_challenge = self._key.sign_challenge(self, challenge)
./crossbar/worker/proxy.py:            signed_challenge = await self._key.sign_challenge(self, challenge, channel_id_type=self.CHANNEL_BINDING)
./crossbar/worker/controller.py:    def sign_challenge(self, challenge: Challenge, channel_id: Optional[bytes], channel_id_type=Optional[str]):
./crossbar/worker/controller.py:        self.log.info('{func}() ...', func=hltype(self.sign_challenge))
./crossbar/worker/controller.py:        result = yield self.call("crossbar.sign_challenge", challenge.method, challenge.extra, channel_id,
./crossbar/worker/controller.py:        self.log.info('{func}(): {result}', func=hltype(self.sign_challenge), result=result)
./crossbar/worker/rlink.py:            signed_challenge = self._router_controller.sign_challenge(challenge, channel_id, channel_id_type)
./crossbar/node/controller.py:    def sign_challenge(self, challenge_method: str, challenge_extra: Dict[str, Any], channel_id_raw: bytes,
./crossbar/node/controller.py:            func=hltype(self.sign_challenge))
./crossbar/node/controller.py:        return self._node.secmod[1].sign_challenge(challenge, channel_id_raw, channel_id_type)
./crossbar/edge/node/management.py:            signed_challenge = self.config.extra['node_key']._sign_challenge(self, challenge)
oberstet commented 2 years ago 
2022-07-03T03:04:41+0200 [Controller 121594] Node key loaded from /tmp/autobahn-python-master/examples/router/.crossbar/key.priv
2022-07-03T03:04:41+0200 [Controller 121594] Traceback (most recent call last):
2022-07-03T03:04:41+0200 [Controller 121594]   File "/home/oberstet/scm/crossbario/crossbar/.tox/py39-examples/bin/crossbar", line 8, in <module>
2022-07-03T03:04:41+0200 [Controller 121594]     sys.exit(run())
2022-07-03T03:04:41+0200 [Controller 121594]   File "/home/oberstet/scm/crossbario/crossbar/crossbar/__init__.py", line 307, in run
2022-07-03T03:04:41+0200 [Controller 121594]     sys.exit(main(executable, args, reactor, personality))
2022-07-03T03:04:41+0200 [Controller 121594]   File "/home/oberstet/scm/crossbario/crossbar/crossbar/node/main.py", line 1275, in main
2022-07-03T03:04:41+0200 [Controller 121594]     options.func(options, reactor=reactor, personality=personality)
2022-07-03T03:04:41+0200 [Controller 121594]   File "/home/oberstet/scm/crossbario/crossbar/crossbar/node/main.py", line 818, in _run_command_start
2022-07-03T03:04:41+0200 [Controller 121594]     node.load_keys(options.cbdir)
2022-07-03T03:04:41+0200 [Controller 121594]   File "/home/oberstet/scm/crossbario/crossbar/crossbar/node/node.py", line 172, in load_keys
2022-07-03T03:04:41+0200 [Controller 121594]     self._node_secmod = SecurityModuleMemory.from_keyfile(os.path.join(cbdir, privfile))
2022-07-03T03:04:41+0200 [Controller 121594]   File "/home/oberstet/scm/crossbario/crossbar/.tox/py39-examples/lib/python3.9/site-packages/autobahn/xbr/_secmod.py", line 501, in from_keyfile
2022-07-03T03:04:41+0200 [Controller 121594]     raise RuntimeError('"private-key-eth" not found in keyfile {}'.format(keyfile))
2022-07-03T03:04:41+0200 [Controller 121594] RuntimeError: "private-key-eth" not found in keyfile /tmp/autobahn-python-master/examples/router/.crossbar/key.priv
main function encountered error
Traceback (most recent call last):
  File "/home/oberstet/scm/crossbario/crossbar/.tox/py39-examples/lib/python3.9/site-packages/twisted/internet/defer.py", line 661, in callback
    self._startRunCallbacks(result)
  File "/home/oberstet/scm/crossbario/crossbar/.tox/py39-examples/lib/python3.9/site-packages/twisted/internet/defer.py", line 763, in _startRunCallbacks
    self._runCallbacks()
  File "/home/oberstet/scm/crossbario/crossbar/.tox/py39-examples/lib/python3.9/site-packages/twisted/internet/defer.py", line 857, in _runCallbacks
    current.result = callback(  # type: ignore[misc]
  File "/home/oberstet/scm/crossbario/crossbar/.tox/py39-examples/lib/python3.9/site-packages/twisted/internet/defer.py", line 1750, in gotResult
    current_context.run(_inlineCallbacks, r, gen, status)
--- <exception caught here> ---
  File "/home/oberstet/scm/crossbario/crossbar/.tox/py39-examples/lib/python3.9/site-packages/twisted/internet/defer.py", line 1660, in _inlineCallbacks
    result = current_context.run(gen.send, result)
  File "/tmp/autobahn-python-master/examples/run-all-examples.py", line 183, in main
    raise RuntimeError("crossbar exited already")
builtins.RuntimeError: crossbar exited already

--------------------------------------------------------------------------------
Running crossbar.io instance
--------------------------------------------------------------------------------
oberstet commented 2 years ago

Bildschirmfoto von 2022-07-04 18-01-05

oberstet commented 2 years ago 
________________________________________________________________ summary ________________________________________________________________
  sphinx: commands succeeded
  flake8: commands succeeded
  mypy: commands succeeded
  yapf: commands succeeded
  bandit: commands succeeded
  py39-pinned-trial: commands succeeded
  py39-unpinned-trial: commands succeeded
  py39-abtrunk-trial: commands succeeded
  py39-examples: commands succeeded
  pytest: commands succeeded
  functests-cb: commands succeeded
oberstet commented 2 years ago

Bildschirmfoto von 2022-07-05 14-03-17 Bildschirmfoto von 2022-07-05 14-10-18

oberstet commented 2 years ago

closing down on unit test issues .. last one from tox -e functests-cfc:

=============================================================== FAILURES ================================================================
__________________________________________________________ test_remote_router ___________________________________________________________

pyfuncitem = <Function 'test_remote_router'>

    def pytest_pyfunc_call(pyfuncitem):
        """Interface to async test call handler."""
        # TODO: only handle 'our' tests?  what is the point of handling others?
        #       well, because our interface allowed people to return deferreds
        #       from arbitrary tests so we kinda have to keep this up for now
>       _run_inline_callbacks(_async_pytest_pyfunc_call, pyfuncitem)

.tox/functests-cfc/lib/python3.9/site-packages/pytest_twisted.py:368: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
.tox/functests-cfc/lib/python3.9/site-packages/pytest_twisted.py:356: in _run_inline_callbacks
    blockon_default(d)
.tox/functests-cfc/lib/python3.9/site-packages/pytest_twisted.py:98: in blockon_default
    result[0].raiseException()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <twisted.python.failure.Failure autobahn.wamp.exception.ApplicationError: ApplicationError(error=<wamp.error.runtime_e...>, args=['realm1'], kwargs={}, enc_algo=None, callee=None, callee_authid=None, callee_authrole=None, forward_for=None)>

    def raiseException(self):
        """
        raise the original exception, preserving traceback
        information if available.
        """
>       raise self.value.with_traceback(self.tb)
E       autobahn.wamp.exception.ApplicationError: ApplicationError(error=<wamp.error.runtime_error>, args=['realm1'], kwargs={}, enc_algo=None, callee=None, callee_authid=None, callee_authrole=None, forward_for=None)

.tox/functests-cfc/lib/python3.9/site-packages/twisted/python/failure.py:500: ApplicationError
____________________________________________________________ test_remote_web ____________________________________________________________

pyfuncitem = <Function 'test_remote_web'>

    def pytest_pyfunc_call(pyfuncitem):
        """Interface to async test call handler."""
        # TODO: only handle 'our' tests?  what is the point of handling others?
        #       well, because our interface allowed people to return deferreds
        #       from arbitrary tests so we kinda have to keep this up for now
>       _run_inline_callbacks(_async_pytest_pyfunc_call, pyfuncitem)

.tox/functests-cfc/lib/python3.9/site-packages/pytest_twisted.py:368: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
.tox/functests-cfc/lib/python3.9/site-packages/pytest_twisted.py:356: in _run_inline_callbacks
    blockon_default(d)
.tox/functests-cfc/lib/python3.9/site-packages/pytest_twisted.py:98: in blockon_default
    result[0].raiseException()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <twisted.python.failure.Failure autobahn.wamp.exception.ApplicationError: ApplicationError(error=<crossbar.error.worke...ng (or starting)"], kwargs={}, enc_algo=None, callee=None, callee_authid=None, callee_authrole=None, forward_for=None)>

    def raiseException(self):
        """
        raise the original exception, preserving traceback
        information if available.
        """
>       raise self.value.with_traceback(self.tb)
E       autobahn.wamp.exception.ApplicationError: ApplicationError(error=<crossbar.error.worker_already_running>, args=["Could not start worker: a worker with ID 'worker1' is already running (or starting)"], kwargs={}, enc_algo=None, callee=None, callee_authid=None, callee_authrole=None, forward_for=None)

.tox/functests-cfc/lib/python3.9/site-packages/twisted/python/failure.py:500: ApplicationError
________________________________________________________ test_remote_web_service ________________________________________________________

pyfuncitem = <Function 'test_remote_web_service'>

    def pytest_pyfunc_call(pyfuncitem):
        """Interface to async test call handler."""
        # TODO: only handle 'our' tests?  what is the point of handling others?
        #       well, because our interface allowed people to return deferreds
        #       from arbitrary tests so we kinda have to keep this up for now
>       _run_inline_callbacks(_async_pytest_pyfunc_call, pyfuncitem)

.tox/functests-cfc/lib/python3.9/site-packages/pytest_twisted.py:368: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
.tox/functests-cfc/lib/python3.9/site-packages/pytest_twisted.py:356: in _run_inline_callbacks
    blockon_default(d)
.tox/functests-cfc/lib/python3.9/site-packages/pytest_twisted.py:98: in blockon_default
    result[0].raiseException()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <twisted.python.failure.Failure autobahn.wamp.exception.ApplicationError: ApplicationError(error=<crossbar.error.worke...ng (or starting)"], kwargs={}, enc_algo=None, callee=None, callee_authid=None, callee_authrole=None, forward_for=None)>

    def raiseException(self):
        """
        raise the original exception, preserving traceback
        information if available.
        """
>       raise self.value.with_traceback(self.tb)
E       autobahn.wamp.exception.ApplicationError: ApplicationError(error=<crossbar.error.worker_already_running>, args=["Could not start worker: a worker with ID 'worker1' is already running (or starting)"], kwargs={}, enc_algo=None, callee=None, callee_authid=None, callee_authrole=None, forward_for=None)

.tox/functests-cfc/lib/python3.9/site-packages/twisted/python/failure.py:500: ApplicationError
================================================= 3 failed, 7 passed in 488.28 seconds ==================================================
2022-07-05T14:22:38+0200 Main loop terminated.
2022-07-05T14:22:38+0200 Unhandled error in Deferred:
2022-07-05T14:22:38+0200 
ERROR: InvocationError for command /bin/sh -c '/home/oberstet/scm/crossbario/crossbar/test/test_setup.sh && pytest -sv --no-install /home/oberstet/scm/crossbario/crossbar/test/functests/cfctests' (exited with code 1)
________________________________________________________________ summary ________________________________________________________________
ERROR:   functests-cfc: commands failed
(cpy39_1) (base) oberstet@intel-nuci7:~/scm/crossbario/crossbar$
oberstet commented 2 years ago

fwiw, from local CI (those are not run in the GH CI), still problems with (though I'm not sure these are related anyways)

tox -e functests-cfc,py39-automate-1,py39-automate-2,py39-xbrnetwork-1

2022-07-05T18:56:21+0200 [Container   76043] 
Traceback (most recent call last):
  File "/home/oberstet/scm/crossbario/crossbar/.tox/py39-automate-2/lib/python3.9/site-packages/twisted/internet/defer.py", line 700, in errback
    self._startRunCallbacks(fail)
  File "/home/oberstet/scm/crossbario/crossbar/.tox/py39-automate-2/lib/python3.9/site-packages/twisted/internet/defer.py", line 763, in _startRunCallbacks
    self._runCallbacks()
  File "/home/oberstet/scm/crossbario/crossbar/.tox/py39-automate-2/lib/python3.9/site-packages/twisted/internet/defer.py", line 857, in _runCallbacks
    current.result = callback(  # type: ignore[misc]
  File "/home/oberstet/scm/crossbario/crossbar/.tox/py39-automate-2/lib/python3.9/site-packages/twisted/internet/defer.py", line 1750, in gotResult
    current_context.run(_inlineCallbacks, r, gen, status)
--- <exception caught here> ---
  File "/home/oberstet/scm/crossbario/crossbar/.tox/py39-automate-2/lib/python3.9/site-packages/twisted/internet/defer.py", line 1656, in _inlineCallbacks
    result = current_context.run(
  File "/home/oberstet/scm/crossbario/crossbar/.tox/py39-automate-2/lib/python3.9/site-packages/twisted/python/failure.py", line 514, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/home/oberstet/scm/crossbario/crossbar/crossbar/master/arealm/arealm.py", line 202, in _check_and_apply
    success = yield self._apply_routercluster_placements(arealm, workergroup, workergroup_placements,
  File "/home/oberstet/scm/crossbario/crossbar/.tox/py39-automate-2/lib/python3.9/site-packages/twisted/internet/defer.py", line 1660, in _inlineCallbacks
    result = current_context.run(gen.send, result)
  File "/home/oberstet/scm/crossbario/crossbar/crossbar/master/arealm/arealm.py", line 790, in _apply_routercluster_placements
    assert role.name not in [
builtins.AssertionError: use of reserved role name "rlink" in role 81469819-9c07-4a72-8e17-60dae70a45ad

________________________________________________________________ summary ________________________________________________________________
  sphinx: commands succeeded
  flake8: commands succeeded
  mypy: commands succeeded
  yapf: commands succeeded
  bandit: commands succeeded
  pytest: commands succeeded
SKIPPED:  py37-pinned-trial: InterpreterNotFound: python3.7
SKIPPED:  py37-unpinned-trial: InterpreterNotFound: python3.7
SKIPPED:  py37-abtrunk-trial: InterpreterNotFound: python3.7
  py39-pinned-trial: commands succeeded
  py39-unpinned-trial: commands succeeded
  py39-abtrunk-trial: commands succeeded
SKIPPED:  py310-pinned-trial: InterpreterNotFound: python3.10
SKIPPED:  py310-unpinned-trial: InterpreterNotFound: python3.10
SKIPPED:  py310-abtrunk-trial: InterpreterNotFound: python3.10
SKIPPED:  pypy38-pinned-trial: InterpreterNotFound: pypy3.8
SKIPPED:  pypy38-unpinned-trial: InterpreterNotFound: pypy3.8
SKIPPED:  pypy38-abtrunk-trial: InterpreterNotFound: pypy3.8
SKIPPED:  py37-examples: InterpreterNotFound: python3.7
  py39-examples: commands succeeded
SKIPPED:  py310-examples: InterpreterNotFound: python3.10
SKIPPED:  pypy38-examples: InterpreterNotFound: pypy3.8
  functests-cb: commands succeeded
ERROR:   functests-cfc: commands failed
  py39-api-1: commands succeeded
SKIPPED:  pypy38-api-1: InterpreterNotFound: pypy3.8
  py39-cli-0: commands succeeded
SKIPPED:  pypy38-cli-0: InterpreterNotFound: pypy3.8
  py39-cli-1: commands succeeded
SKIPPED:  pypy38-cli-1: InterpreterNotFound: pypy3.8
  py39-cli-2: commands succeeded
SKIPPED:  pypy38-cli-2: InterpreterNotFound: pypy3.8
  py39-cli-3: commands succeeded
SKIPPED:  pypy38-cli-3: InterpreterNotFound: pypy3.8
ERROR:   py39-automate-1: commands failed
SKIPPED:  pypy38-automate-1: InterpreterNotFound: pypy3.8
ERROR:   py39-automate-2: keyboardinterrupt
ERROR:   pypy38-automate-2: undefined
ERROR:   py39-xbrnetwork-1: undefined
ERROR:   pypy38-xbrnetwork-1: undefined