rare-freertr / freeRtr-containerlab

RARE/freeRtr for Containerlab Network Operating System networks simulator
Apache License 2.0
11 stars 3 forks source link

IPv6 OOB does not work #10

Closed Takalele closed 4 weeks ago

Takalele commented 1 month ago

Hi,

the issue with IPv6 out-of-band connectivity stems from the line cfg.Sysctls["net.ipv6.conf.all.disable_ipv6"] = "1" in rare.go.

I believe it’s unnecessary to enforce this setting within the kind itself, as disabling IPv6 within the container should suffice.

To fix this, we could remove the line cfg.Sysctls["net.ipv6.conf.all.disable_ipv6"] = "1" from the kind configuration. Additionally, in the hwdet-mgmt.sh script, we should update the following:

from:

IPv4=`hostname -i | awk '{print $1}'`
IPv6=`hostname -i | awk '{print $2}'`

to:

IPv4=`hostname -i | awk '{print $2}'`
IPv6=`hostname -i | awk '{print $1}'`

BR Takalele

frederic-loui commented 1 month ago

That's a valid feedback. The general approach is that during installation freeRtr takes full control of the networking stack. As you have already noticed, freeRtr can run and be connected to different dataplane. In containerlab flavor, software dataplane is used. The reason ipv6 has been disabled is to let freeRtr full control on the IPv6 stack. In that context, one solution is to disable ipv6 for all interface except the OOB interface. Feel free to submet a PR. I'd be glad to merge it.

All welcome to freeRtr networking community !

Takalele commented 1 month ago

during my testing, I found that the only change needed to get IPv6 OOB working again is to comment out line 41 in the rare kind. It is sufficient to disable IPv6 via sysctl during the container build process, as done by lines 43 and 44 in the install-rtr.sh script. With this change, Docker is still able to delegate the IPv6 address to the container, as the hostname -i command returns both IPv4 and IPv6 addresses, even though IPv6 remains globally disabled.

Is there anything I might have overlooked? If this solution is acceptable to you, I would be happy to make the necessary commits in both the containerlab and freeRtr-containerlab repositories.

Alternatively, one could try to modify rare.go to disable IPv6 via cfg.Sysctls for all interfaces except eth0. However, it is unclear whether this is possible and would work as expected.

Also, thank you for the rare SOHO blog – it was very helpful for getting started with FreeRTR three years ago!

takalele@ct-labs:~$ docker ps
CONTAINER ID   IMAGE                         COMMAND                  CREATED      STATUS      PORTS     NAMES
bf5dffe4b0ae   freertr-containerlab:latest   "/bin/sh -c /rtr/sta…"   3 days ago   Up 3 days             clab-rtr-l2transport-internet
74afc881a493   freertr-containerlab:latest   "/bin/sh -c /rtr/sta…"   3 days ago   Up 3 days             clab-rtr-l2transport-rtr1
18ec8dab6a87   alpine:latest                 "/bin/sh"                3 days ago   Up 3 days             clab-rtr-l2transport-n2
cba7511f8f85   alpine:latest                 "/bin/sh"                3 days ago   Up 3 days             clab-rtr-l2transport-n1
4581ec359ab2   freertr-containerlab:latest   "/bin/sh -c /rtr/sta…"   3 days ago   Up 3 days             clab-rtr-l2transport-rtr2

takalele@ct-labs:~$ docker exec -it clab-rtr-l2transport-internet bash
root@rtr1:~# sysctl net.ipv6.conf.all.disable_ipv6
net.ipv6.conf.all.disable_ipv6 = 1
root@rtr1:~# hostname -i
2001:172:20:20::2 172.20.20.2

root@rtr1:~# ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether 02:42:ac:14:14:02  txqueuelen 0  (Ethernet)
        RX packets 36820  bytes 2938278 (2.8 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 29233  bytes 2666730 (2.5 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth1: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST>  mtu 1500
        ether aa:c1:ab:eb:1a:19  txqueuelen 0  (Ethernet)
        RX packets 239845  bytes 47982296 (45.7 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 444002  bytes 69025459 (65.8 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth2: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST>  mtu 1500
        ether aa:c1:ab:97:e1:56  txqueuelen 0  (Ethernet)
        RX packets 17441  bytes 2310696 (2.2 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 17439  bytes 2310536 (2.2 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth3: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST>  mtu 1500
        ether aa:c1:ab:85:9c:6e  txqueuelen 0  (Ethernet)
        RX packets 17439  bytes 2310536 (2.2 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 17441  bytes 2310696 (2.2 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth4: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST>  mtu 1500
        ether aa:c1:ab:53:b5:a3  txqueuelen 0  (Ethernet)
        RX packets 1518  bytes 343347 (335.2 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 20398  bytes 2745908 (2.6 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65535
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 841505  bytes 158500268 (151.1 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 841505  bytes 158500268 (151.1 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

takalele@ct-labs:~$ telnet clab-rtr-l2transport-rtr1
Trying 2001:172:20:20::2...
Connected to clab-rtr-l2transport-rtr1.
Escape character is '^]'.
welcome
username:rare
password:
line ready
rtr1#show ipv6 interface
interface    state  address                    netmask
ethernet1    up     fe80::a8c1:abff:fe58:3c95  ffff:ffff:ffff:ffff::
ethernet255  up     2001:172:20:20::2          ffff:ffff:ffff:ffff::

rtr1#ping 2001:172:20:20::1 vrf OOB
pinging 2001:172:20:20::1, src=null, vrf=OOB, cnt=5, len=64, df=false, tim=1000, gap=0, ttl=255, tos=0, sgt=0, flow=0, fill=0, alrt=-1, sweep=false, multi=false
!!!!!
result=100.0%, recv/sent/lost/err=5/5/0/0, took 2, min/avg/max/dev rtt=0/0.2/1/0.1, ttl 64/64.0/64/0.0, tos 0/0.0/0/0.0
Takalele commented 1 month ago

@frederic-loui I've made updates to the rare kind to disable IPv6 at runtime, first for all interfaces and then globally. This gives Docker enough time to assign the IPv6 address to the container, ensuring that the hostname -i command captures the correct address. If this approach is acceptable to you, I would like to proceed with creating a pull request in the containerlab repository.

the updated rare kind

frederic-loui commented 1 month ago

I would not modify rare kind and leave the current containerlab internal behavior WRT OOB management interface.

In the bash script, I'd rather: 1- List all interfaces explicitely eth[0-9]+

ls -d /proc/sys/net/ipv6/conf/eth*

2- For all eth* interfaces disable IPv6 except eth255:

echo 1 | sudo tee /proc/sys/net/ipv6/conf/eth*/disable_ipv6
echo 0 | sudo tee /proc/sys/net/ipv6/conf/eth255/disable_ipv6
# ... Or use sysctl equivalent or sysctl.conf 

Well this is a suggestion ! Maybe I might have overlooked another problem.

PS: Thanks for the feedback regarding the blog. The project is still alive and kicking but moving at a lower pace because of various reasons. However, your feedback is invaluable as we don't know who is using our work/stuff and somehow we think that our doc is "not read"😄 Obviously, please do not hesitate to provide suggestion, improvement. We are always willing to help. It is just that we are helping at the pace we can follow at some point in time. We plan to refresh these articles with current technology, things are evolving so fast that the docs updates cannot keep up... 😞

Takalele commented 1 month ago

I’ve tested this, but unfortunately, I haven’t found a way to get IPv6 OOB working without modifying the rare kind.

The simplest solution would be to comment out line 41: cfg.Sysctls["net.ipv6.conf.all.disable_ipv6"] = "1". In my opinion, there’s no need to disable net.ipv6.conf.all.disable_ipv6 using the Docker sysctl flag, as IPv6 will be disabled by hwdet-all.sh during FreeRTR startup. Since IPv6 is disabled within the container and not via Docker, the Docker daemon is able to assign an IPv6 address to the container. How do we want to procced?

Regarding the blog, where would you like me to submit some suggestions or feedback? Do you have an email address or a contact form I can use?

frederic-loui commented 1 month ago

Thanks @Takalele for your feedback and tests. We should keep it simple as you suggested. Feel free to submit a PR re-enabling back IPv6. WRT to the blog, you reach me via email (frederic.loui@gmail.com)

Takalele commented 4 weeks ago

merged