search

Giuseppe1992 / Distrinet

Distributed Network emulator, based on Mininet
MIT License
17 stars 16 forks source link

Debian support #54

Closed t-valette closed 4 years ago

t-valette commented 4 years ago

I'm having some problems running distrinet on Debian 10 Buster (bin/dmn hangs at the "Adding switches" step). FYI : I had to install by hand lxd with snap and cryptography==2.9 with pip3

In this context : gros-95 => Client gros-96 => Master gros-98 and gros-99 => Workers

root@gros-95:~/Distrinet/mininet# python3 bin/dmn -v debug --bastion=gros-96 --workers="gros-96,gros-98,gros-99" --controller=lxcremote,ip=192.168.0.1 --topo=linear,2
*** WARNING: Experimental cloud mode!
*** Using LxcNode, LxcOVSSwitch, CloudLink
*** Mapping: {'h1': 'gros-96', 'h2': 'gros-98', 's1': 'gros-99', 's2': 'gros-96'}
*** STARTING DISTRINET ****** Creating network
*** Adding controller
Connecting to remote controller at 192.168.0.1:6653
*** Adding hosts:
h1 h2
*** Adding switches:

added intf lo (0) to node s1
*** s1 : ('ifconfig', 'lo', 'up')
(s1 exited - ignoring cmd('ifconfig', 'lo', 'up'))
s1
added intf lo (0) to node s2
*** s2 : ('ifconfig', 'lo', 'up')
(s2 exited - ignoring cmd('ifconfig', 'lo', 'up'))
s2
root@gros-96:~# ansible all -m raw -a 'lxc ls'
127.0.0.1 | CHANGED | rc=0 >>
+------+-------+------+------+------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+------+-------+------+------+------+-----------+

gros-98 | CHANGED | rc=0 >>
+------+-------+------+------+------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+------+-------+------+------+------+-----------+
Shared connection to gros-98 closed.

gros-99 | CHANGED | rc=0 >>
+------+-------+------+------+------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+------+-------+------+------+------+-----------+
Shared connection to gros-99 closed.
root@gros-96:~# ansible all -m raw -a 'lxc image ls'
127.0.0.1 | CHANGED | rc=0 >>
+--------+--------------+--------+------------------------------------+--------------+-----------+----------+------------------------------+
| ALIAS  | FINGERPRINT  | PUBLIC |            DESCRIPTION             | ARCHITECTURE |   TYPE    |   SIZE   |         UPLOAD DATE          |
+--------+--------------+--------+------------------------------------+--------------+-----------+----------+------------------------------+
| switch | 5448ace25988 | yes    | Ubuntu 18.04 LTS server (20190514) | x86_64       | CONTAINER | 278.98MB | Apr 21, 2020 at 8:17am (UTC) |
+--------+--------------+--------+------------------------------------+--------------+-----------+----------+------------------------------+
| ubuntu | f5a783211484 | yes    | Ubuntu 18.04 LTS server (20190514) | x86_64       | CONTAINER | 267.68MB | Apr 21, 2020 at 8:17am (UTC) |
+--------+--------------+--------+------------------------------------+--------------+-----------+----------+------------------------------+

gros-98 | CHANGED | rc=0 >>
+--------+--------------+--------+------------------------------------+--------------+-----------+----------+------------------------------+
| ALIAS  | FINGERPRINT  | PUBLIC |            DESCRIPTION             | ARCHITECTURE |   TYPE    |   SIZE   |         UPLOAD DATE          |
+--------+--------------+--------+------------------------------------+--------------+-----------+----------+------------------------------+
| switch | 5448ace25988 | yes    | Ubuntu 18.04 LTS server (20190514) | x86_64       | CONTAINER | 278.98MB | Apr 21, 2020 at 8:22am (UTC) |
+--------+--------------+--------+------------------------------------+--------------+-----------+----------+------------------------------+
| ubuntu | f5a783211484 | yes    | Ubuntu 18.04 LTS server (20190514) | x86_64       | CONTAINER | 267.68MB | Apr 21, 2020 at 8:22am (UTC) |
+--------+--------------+--------+------------------------------------+--------------+-----------+----------+------------------------------+
Shared connection to gros-98 closed.

gros-99 | CHANGED | rc=0 >>
+--------+--------------+--------+------------------------------------+--------------+-----------+----------+------------------------------+
| ALIAS  | FINGERPRINT  | PUBLIC |            DESCRIPTION             | ARCHITECTURE |   TYPE    |   SIZE   |         UPLOAD DATE          |
+--------+--------------+--------+------------------------------------+--------------+-----------+----------+------------------------------+
| switch | 5448ace25988 | yes    | Ubuntu 18.04 LTS server (20190514) | x86_64       | CONTAINER | 278.98MB | Apr 21, 2020 at 8:22am (UTC) |
+--------+--------------+--------+------------------------------------+--------------+-----------+----------+------------------------------+
| ubuntu | f5a783211484 | yes    | Ubuntu 18.04 LTS server (20190514) | x86_64       | CONTAINER | 267.68MB | Apr 21, 2020 at 8:22am (UTC) |
+--------+--------------+--------+------------------------------------+--------------+-----------+----------+------------------------------+
Shared connection to gros-99 closed.
root@gros-96:~# ansible all -m raw -a 'ifconfig'
127.0.0.1 | CHANGED | rc=0 >>
admin-br: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.0.1  netmask 255.0.0.0  broadcast 192.255.255.255
        inet6 fe80::6884:ff:fe1e:4cbf  prefixlen 64  scopeid 0x20<link>
        ether 6a:84:00:1e:4c:bf  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 3998  bytes 171104 (167.0 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

br0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 172.16.66.96  netmask 255.255.240.0  broadcast 172.16.79.255
        inet6 fe80::9a03:9bff:feb0:971a  prefixlen 64  scopeid 0x20<link>
        ether 98:03:9b:b0:97:1a  txqueuelen 1000  (Ethernet)
        RX packets 348060  bytes 981853073 (936.3 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 165567  bytes 1195585085 (1.1 GiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eno1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether 98:03:9b:b0:97:1a  txqueuelen 1000  (Ethernet)
        RX packets 1007626  bytes 1032350560 (984.5 MiB)
        RX errors 0  dropped 431  overruns 0  frame 0
        TX packets 921918  bytes 1245524691 (1.1 GiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eno2: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::9a03:9bff:feb0:971b  prefixlen 64  scopeid 0x20<link>
        ether 98:03:9b:b0:97:1b  txqueuelen 1000  (Ethernet)
        RX packets 61064  bytes 4829470 (4.6 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 137  bytes 10323 (10.0 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 10251  bytes 705418 (688.8 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 10251  bytes 705418 (688.8 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

gros-98 | CHANGED | rc=0 >>
br0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 172.16.66.98  netmask 255.255.240.0  broadcast 172.16.79.255
        inet6 fe80::9a03:9bff:feb0:acde  prefixlen 64  scopeid 0x20<link>
        ether 98:03:9b:b0:ac:de  txqueuelen 1000  (Ethernet)
        RX packets 236838  bytes 970883314 (925.9 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 99205  bytes 8193495 (7.8 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eno1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether 98:03:9b:b0:ac:de  txqueuelen 1000  (Ethernet)
        RX packets 888791  bytes 1019322378 (972.1 MiB)
        RX errors 0  dropped 430  overruns 0  frame 0
        TX packets 99238  bytes 8210037 (7.8 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eno2: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::9a03:9bff:feb0:acdf  prefixlen 64  scopeid 0x20<link>
        ether 98:03:9b:b0:ac:df  txqueuelen 1000  (Ethernet)
        RX packets 60681  bytes 4855139 (4.6 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 157  bytes 11368 (11.1 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 8664  bytes 433896 (423.7 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 8664  bytes 433896 (423.7 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Shared connection to gros-98 closed.

gros-99 | CHANGED | rc=0 >>
br0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 172.16.66.99  netmask 255.255.240.0  broadcast 172.16.79.255
        inet6 fe80::9a03:9bff:feb0:a74e  prefixlen 64  scopeid 0x20<link>
        ether 98:03:9b:b0:a7:4e  txqueuelen 1000  (Ethernet)
        RX packets 227640  bytes 970411116 (925.4 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 97731  bytes 8087482 (7.7 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eno1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether 98:03:9b:b0:a7:4e  txqueuelen 1000  (Ethernet)
        RX packets 883490  bytes 1018978777 (971.7 MiB)
        RX errors 0  dropped 431  overruns 0  frame 0
        TX packets 97761  bytes 8103726 (7.7 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eno2: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::9a03:9bff:feb0:a74f  prefixlen 64  scopeid 0x20<link>
        ether 98:03:9b:b0:a7:4f  txqueuelen 1000  (Ethernet)
        RX packets 60695  bytes 4853506 (4.6 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 168  bytes 12004 (11.7 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 8664  bytes 433896 (423.7 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 8664  bytes 433896 (423.7 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Shared connection to gros-99 closed.
dsaucez commented 4 years ago

Can you try with the latest commit 2cdb162 ? It works for a Debian:10 container in docker (but hosts are ubuntu:18.04).

t-valette commented 4 years ago

I reinstalled distrinet on the client and I still have the same issue even with the latest commit (bin/dmn hangs at the "Adding switches" step)

dsaucez commented 4 years ago

What distrib on the hosts?

t-valette commented 4 years ago

They are all running Debian 10 Buster

Giuseppe1992 commented 4 years ago

Seems that lxd and ansible are working correctly. I am trying with Debian on G5K, I think there is a problem with the ssh connections.

dsaucez commented 4 years ago

I am on it.

dsaucez commented 4 years ago

@Kyominii The problem comes from using snap and the issue that lxc and lxd did not appear in the PATH.

The current fix is a bit dirty but it works with Debian 10 for when you setup from scratch a node.

To setup the cluster, instead of using install-lxd.yml you have to use install-lxd-debian-10.yml:

ansible-playbook ~/install-lxd-debian-10.yml

It will use snap instead of apt to install LXD and will create a symbolic link to make sure lxc and lxd commandes are reachable. The issue comes from asyncssh that doesn't seem to properly load the environment variables.

If your machines are already setup, just make sure that lxc and lxd commands appear in the PATH environment variable.

A clean solution will be found when #55 will be closed, using latest LXD version.

Giuseppe1992 commented 4 years ago

@Kyominii The problem comes from using snap and the issue that lxc and lxd did not appear in the PATH.

The current fix is a bit dirty but it works with Debian 10 for when you setup from scratch a node.

To setup the cluster, instead of using install-lxd.yml you have to use install-lxd-debian-10.yml:

ansible-playbook ~/install-lxd-debian-10.yml

It will use snap instead of apt to install LXD and will create a symbolic link to make sure lxc and lxd commandes are reachable. The issue comes from asyncssh that doesn't seem to properly load the environment variables.

If your machines are already setup, just make sure that lxc and lxd commands appear in the PATH environment variable.

A clean solution will be found when #55 will be closed, using latest LXD version.

with #55 DIstrinet installs automatically LXD4.0/stable, directly with the default ansible playbook; You can use install-lxd.yml for Ubuntu18.04 and Debian10.

ansible-playbook ~/install-lxd.yml

Should work now

dsaucez commented 4 years ago

@Kyominii The problem comes from using snap and the issue that lxc and lxd did not appear in the PATH. The current fix is a bit dirty but it works with Debian 10 for when you setup from scratch a node. To setup the cluster, instead of using install-lxd.yml you have to use install-lxd-debian-10.yml:

ansible-playbook ~/install-lxd-debian-10.yml

It will use snap instead of apt to install LXD and will create a symbolic link to make sure lxc and lxd commandes are reachable. The issue comes from asyncssh that doesn't seem to properly load the environment variables. If your machines are already setup, just make sure that lxc and lxd commands appear in the PATH environment variable. A clean solution will be found when #55 will be closed, using latest LXD version.

with #55 DIstrinet installs automatically LXD4.0/stable, directly with the default ansible playbook; You can use install-lxd.yml for Ubuntu18.04 and Debian10.

ansible-playbook ~/install-lxd.yml

Should work now

@Kyominii please use commit 85cdc3044426271f175f5e2a12ea7a26174cc521 if you want a stable one and stick to ansible-playbook ~/install-lxd-debian-10.yml, the rest is experimental and has not been extensively tested yet.

dsaucez commented 4 years ago

@Kyominii we made little changes as you can see.

For the stable versions you can use install-lxd.yml on Ubuntu 18.04 and install-lxd-debian-10.yml on Debian 10.

@Giuseppe1992 also made a version that can be used on either Ubuntu or Debian and that is called install-lxd-snap.yml. For it you must have ansible (version>=2.8) on the master. It will automatically install snap on the workers and use it to install LXD. Actually we would happy to have your feedback on using that new proposition as the objective is to eventually move to snap and LXD 4.

t-valette commented 4 years ago

I have some issue when ansible tries to install a snap package (core or lxd)

root@gros-72:~# snap install hello-world
error: cannot perform the following tasks:
- Fetch and check assertions for snap "core" (8935) (cannot get device session from store: store server returned status 400 and body "{\"error_list\":[{\"code\":\"invalid-assertion\",\"message\":\"invalid assertion: could not validate model assertion (revision 0 is already the current revision)\"}],\"errors\":[\"invalid assertion: could not validate model assertion (revision 0 is already the current revision)\"],\"result\":\"error\"}\n")

I'm not sure if it related to snap itself or the installation of snapd

EDIT : It was related to snapd, it's working now

t-valette commented 4 years ago

I manage to make Distrinet running under Debian 10 (client + master + workers) using install-lxd-snap.yml without any trouble (except the snapd issue, but it was temporary)

dsaucez commented 4 years ago

EDIT : It was related to snapd, it's working now

What was exactly the problem? Does it come from our tutorial + playbooks or was it an error on your side?

t-valette commented 4 years ago

It was probably on my side (it looked like a network issue), I will retry later this afternoon to confirm it works with no problem now

t-valette commented 4 years ago

It's all good, I can use dmn without any trouble using install-lxd-snap.yml with debian 10