error: Umbra Experiment Error after begin command in umbra-cli

[nimaafraz]

After running the begin command in umbra-cli an error occurs:

:umbra> begin

: Beginning :

-> task: Experiment Begin
-> error: Umbra Experiment Error

nimaafraz@umbra-1:/tmp/umbra/logs$ cat umbracliapp-umbra-cli.log
.
.
.
2021-06-15 17:07:07,529 [INFO] umbra.cli.main: begin triggered
2021-06-15 17:07:07,535 [DEBUG] umbra.design.basis: Could not look for profile where nodes type None
2021-06-15 17:07:07,536 [DEBUG] umbra.cli.interfaces: Exception: Could not reach stub channel - exception ConnectionRefusedError(111, "Connect call failed ('127.0.0.1', 8956)")
2021-06-15 17:07:07,539 [INFO] umbra.cli.main: Could not reach stub channel - exception ConnectionRefusedError(111, "Connect call failed ('127.0.0.1', 8956)")

Looking at the aux the broker does not seem to be running:

nimaafraz@umbra-1:/tmp/umbra/logs$ ps aux | grep umbra
root       34135  0.0  0.0  11184  4812 pts/0    S+   17:03   0:00 sudo umbra-cli --uuid umbra-cli --address 127.0.0.1:9988 --debug --source /tmp/umbra/fabric/
root       34136  0.2  0.3  77024 54432 pts/0    S+   17:03   0:01 /usr/bin/python3.8 /usr/local/bin/umbra-cli --uuid umbra-cli --address 127.0.0.1:9988 --debug --source /tmp/umbra/fabric/
nimaafr+   41208  0.0  0.0   8168  2572 pts/1    S+   17:12   0:00 grep --color=auto umbra

[andrekuros]

Same error here. Seams like, despite the example is running on fabric, they are looking for iroha plugins during the load:

fabric@kuros-Virtual-Machine:/home/kuros/umbra/examples$ umbra-broker --uuid default-broker --address 127.0.0.1:8956 --debug

Traceback (most recent call last):
  File "/usr/local/bin/umbra-broker", line 7, in <module>
    exec(compile(f.read(), __file__, 'exec'))
  File "/tmp/umbra/source/umbra/broker/umbra-broker", line 7, in <module>
    from umbra.broker.main import Broker
  File "/tmp/umbra/source/umbra/broker/main.py", line 6, in <module>
    from umbra.broker.operator import Operator
  File "/tmp/umbra/source/umbra/broker/operator.py", line 21, in <module>
    from umbra.broker.plugins.iroha import IrohaEvents
  File "/tmp/umbra/source/umbra/broker/plugins/iroha.py", line 7, in <module>
    from iroha.primitive_pb2 import can_set_my_account_detail
ModuleNotFoundError: No module named 'iroha'

[andrekuros]

After comment the 2 lines in "/tmp/umbra/source/umbra/broker/operator.py" that call iroha libs, the broker runs, however the experiment keep failing.

root       11660  0.0  0.0  20708  4548 pts/1    S+   13:07   0:00 sudo umbra-cli --uuid umbra-cli --address 127.0.0.1:9988 --debug --source /tmp/umbra/fabric/
root       11662  0.1  0.6  86360 55016 pts/1    S+   13:07   0:01 /usr/bin/python3.8 /usr/local/bin/umbra-cli --uuid umbra-cli --address 127.0.0.1:9988 --debug --source /tmp/umbra/fabric/
root       32078 30.6  0.6 137164 60220 pts/1    S+   13:31   0:00 /usr/bin/python3.8 /usr/local/bin/umbra-broker --uuid default-broker --address 127.0.0.1:8956 --debug
root       32080 18.6  0.5  77932 46476 pts/1    S+   13:31   0:00 /usr/bin/python3.8 /usr/local/bin/umbra-monitor --uuid default-monitor --address 127.0.0.1:8958 --debug
root       32082  0.0  0.0  20952  4740 pts/1    S+   13:31   0:00 sudo umbra-scenario --uuid default-scenario --address 127.0.0.1:8957 --debug
root       32083 20.6  0.5  80200 48720 pts/1    S+   13:31   0:00 /usr/bin/python3.8 /usr/local/bin/umbra-scenario --uuid default-scenario --address 127.0.0.1:8957 --debug
fabric     32209  0.0  0.0  17668   736 pts/0    S+   13:31   0:00 grep --color=auto umbra

Experiment:

2021-06-16 13:35:47,701 [INFO] umbra.cli.main: Executing commands: ['begin']
2021-06-16 13:35:47,701 [INFO] umbra.cli.main: begin triggered
2021-06-16 13:35:47,704 [DEBUG] umbra.design.basis: Could not look for profile where nodes type None
2021-06-16 13:35:47,710 [DEBUG] umbra.cli.interfaces: Exception: Could not reach stub grpcerror - exception GRPCError(<Status.UNKNOWN: 2>, 'Internal Server Error', None)
2021-06-16 13:35:47,711 [INFO] umbra.cli.main: Could not reach stub grpcerror - exception GRPCError(<Status.UNKNOWN: 2>, 'Internal Server Error', None)

[nimaafraz]

We are currently working on the issue with the Fabric-Iroha dependencies. For the time being, you could just install the dependencies for both before running any experiments. Regarding the grpcerror, I have been trying to update the dependencies grpclib and grpcio to see if the problem is resolved but the error still is there. It seems that evenafter updating grpclib to 0.4.1 python still uses a 0.3.2 version:

2021-06-16 15:39:42,910 [DEBUG] umbra.design.basis: Node switch s1 data: {'env': 'umbra-default', 'environment': 'umbra-default', 'lifecycle': None, 'links': {'org1': {'org': 'org1', 'profile': 'links', 'type': 'internal'}, 'org2': {'org': 'org2', 'profile': 'links', 'type': 'internal'}}, 'name': 's1', 'profile': None, 'resources': {}, 'tun_ids': 1, 'type': 'switch'}
2021-06-16 15:39:42,910 [ERROR] grpclib.server: Application error
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/grpclib-0.3.2-py3.8.egg/grpclib/server.py", line 440, in request_handler
    await method_func(stream)
  File "/tmp/umbra/source/umbra/broker/main.py", line 22, in Execute
    reply = await self.operator.execute(request)
  File "/tmp/umbra/source/umbra/broker/operator.py", line 281, in execute
    info, error = await self.start(uid)
  File "/tmp/umbra/source/umbra/broker/operator.py", line 240, in start
    acks, stats = await self.call_scenarios(uid, topology, "start")
  File "/tmp/umbra/source/umbra/broker/operator.py", line 190, in call_scenarios
    topo_envs = topology.build_environments()
  File "/tmp/umbra/source/umbra/design/basis.py", line 523, in build_environments
    if n not in envs[node_env]["nodes"]:
KeyError: None

[nimaafraz]

Ok, I think I have a lead. So I update grpclib to 0.4.1 in the setup.py but after running the install command in umbra-cli it again installs the older version of grpclib 0.3.2 and defauls on that one. I will try to change this.

:umbra> install

: Installing :

-> task: Installing Umbra at environment umbra-default
-> result: Install Umbra in environment umbra-default Ok

2021-06-17 07:54:42,158 [INFO] umbra.cli.envs: Implementing actions - ['install']
2021-06-17 07:54:42,159 [INFO] umbra.cli.envs: Implementing action - install
2021-06-17 07:54:42,159 [INFO] umbra.cli.envs: Calling action - install - Installing environment umbra-default
2021-06-17 07:54:42,161 [INFO] umbra.cli.envs: Workflow install
2021-06-17 07:54:42,161 [INFO] umbra.cli.envs: Workflow source files - action install
2021-06-17 07:54:42,161 [INFO] umbra.cli.envs: Source files - /tmp/umbra/source has files: ['umbra', 'deps', 'docs', 'umbra.egg-info', '.git', 'build', 'examples', 'aux']
2021-06-17 07:54:42,162 [INFO] umbra.cli.envs: Umbra source files already cloned to: /tmp/umbra/source
2021-06-17 07:54:42,162 [INFO] umbra.cli.envs: Executing command - install requirements, deps, umbra install
2021-06-17 07:54:42,162 [INFO] umbra.cli.envs: Running local process cd /tmp/umbra/source && sudo apt install -y make && sudo make requirements install-deps install
2021-06-17 07:55:20,017 [INFO] umbra.cli.envs: Process Status CompletedProcess(args='cd /tmp/umbra/source && sudo apt install -y make && sudo make requirements install-deps install', returncode=0)
2021-06-17 07:55:20,017 [INFO] umbra.cli.envs: Process stdout ok
2021-06-17 07:55:20,017 [INFO] umbra.cli.envs: Workflow model fabric - action install
2021-06-17 07:55:20,017 [INFO] umbra.cli.envs: Running local process cd /tmp/umbra/source && sudo make install-fabric
2021-06-17 07:55:36,656 [INFO] umbra.cli.envs: Process Status CompletedProcess(args='cd /tmp/umbra/source && sudo make install-fabric', returncode=0)
2021-06-17 07:55:36,656 [INFO] umbra.cli.envs: Process stdout ok
2021-06-17 07:55:36,656 [INFO] umbra.cli.envs: Stats: True - msg: ok
2021-06-17 07:55:36,659 [INFO] umbra.cli.envs: Replying messages
2021-06-17 07:55:36,659 [INFO] umbra.cli.envs: [None]