search

cnp3 / ipmininet

Mininet extension to make experimenting with IP networks easy
GNU General Public License v2.0
65 stars 51 forks source link

[OpenR] Enhance the install script for OpenR #30

Closed butjar closed 5 years ago

butjar commented 5 years ago

Enhance the install script to build and install OpenR. To run OpenR in ipmininet "routers" there are two main requirements for the host system:

  1. OpenR and its dependencies are installed
  2. IPv6 is enabled (disabled by the mininet install script)

The OpenR build requires several steps:

  1. Clone OpenR
  2. Generate the build script from python scripts
  3. Make the generated build script executable
  4. Build and install OpenR using the build script

IPv6 support is re-enabled by changing/ removing the kernel parameters in /etc/default/grub and /etc/sysctl.conf.

jadinm commented 5 years ago

Thanks for the PR!

However, I was unable to execute it correctly on a Ubuntu 16.04 VM. It seems that 3GB is not sufficient to run. The building of the OpenR library fails because gcc is killed by my VM due to the lack of memory.

How much RAM do you have when you compile OpenR?

butjar commented 5 years ago

Thanks for the PR!

However, I was unable to execute it correctly on a Ubuntu 16.04 VM. It seems that 3GB is not sufficient to run. The building of the OpenR library fails because gcc is killed by my VM due to the lack of memory.

I know it's huge unfortunately. I didn't measure, but I think it takes more than an hour with all dependencies on a single core.

How much RAM do you have when you compile OpenR?

For me 4 GB work fine.

Do you suggest packaging it though? We can disable it by default (-a).

butjar commented 5 years ago

Re-enabling IPv6 should be valuable for ipmininet anyway.

jadinm commented 5 years ago

Do you suggest packaging it though? We can disable it by default (-a).

I don't think there is an OpenR package and I don't think that we should maintain such package. So yes, I would remove it from the default build (-a).

I would also put in the documentation of the options (in the parse_args() function) that:

Re-enabling IPv6 should be valuable for ipmininet anyway.

Agreed

butjar commented 5 years ago

:+1: I will update the PR

jadinm commented 5 years ago

I will test it again later ;)

Also, I have a question. Which distributions are supported by OpenR ? Based on the name of the script debian_system_builder.py, I guess that only Ubuntu and Debian are supported but not Fedora.

If this is the case, you should check the distrib and only install if Ubuntu and Debian with something like:

if dist.NAME != "Ubuntu" and dist.NAME != "Debian":
  print("We only support OpenR installation on Ubuntu and Debian")
else:
  install_openr()
butjar commented 5 years ago

I will test it again later ;)

Also, I have a question. Which distributions are supported by OpenR ? Based on the name of the script debian_system_builder.py, I guess that only Ubuntu and Debian are supported but not Fedora.

Yes, the build script uses apt to install dependencies. The README says they test on ubuntu16.04. I created the build script generator since the official build toolchain was docker only. :thinking: Maybe I can have a look if replacing the package manager calls is sufficient for a fedora_system_builder.py.

If this is the case, you should check the distrib and only install if Ubuntu and Debian with something like:

if dist.NAME != "Ubuntu" and dist.NAME != "Debian":
  print("We only support OpenR installation on Ubuntu and Debian")
else:
  install_openr()

Good point.

jadinm commented 5 years ago

I tested it. It works on Ubuntu 16.04. However, the script does not work in Debian Stretch. The installation script tries to install gcc-5 which does not exist in this distribution.

butjar commented 5 years ago

I tested it. It works on Ubuntu 16.04. However, the script does not work in Debian Stretch. The installation script tries to install gcc-5 which does not exist in this distribution.

Cool, thanks. The gcc version can be configured in the script. I will try to make it work on Debain as well. Thanks for the hint.

butjar commented 5 years ago

@jadinm Could you point me to the Debian and Fedora image you are testing against?

jadinm commented 5 years ago

I use the vagrant boxes bento/debian-9 and bento/fedora-28.

oliviertilmans commented 5 years ago
  • OpenR requires 4GB of RAM to build (because it is a bit frustrating to discover it after an hour ^^)

Does it use LTO? The fact that it takes an hour to crash hints towards a link-time error. Disabling LTO/Enabling ThinLTO could help there (if LTO is used at all) at the expense of some slightly reduced perf. for openrd (nothing that matters here as we won't emulate networks of 1k nodes...).

butjar commented 5 years ago
  • OpenR requires 4GB of RAM to build (because it is a bit frustrating to discover it after an hour ^^)

Does it use LTO? The fact that it takes an hour to crash hints towards a link-time error. Disabling LTO/Enabling ThinLTO could help there (if LTO is used at all) at the expense of some slightly reduced perf. for openrd (nothing that matters here as we won't emulate networks of 1k nodes...).

I think it doesn't. However, I think the reason of the long build time results from more than 10 dependencies which are built from scratch as well. So there are several builds which could be probably optimized :disappointed:.

butjar commented 5 years ago

I will adjust this PR to build OpenR for a Ubuntu VM only, but we can draw two issues from here:

butjar commented 5 years ago

I tested it. It works on Ubuntu 16.04. However, the script does not work in Debian Stretch. The installation script tries to install gcc-5 which does not exist in this distribution.

Tested on Debian also with 'gcc-6', but fails at one of the dependencies.

butjar commented 5 years ago

I use the vagrant boxes bento/debian-9 and bento/fedora-28.

If multi-OS portability is important https://kitchen.ci/ is also a neat tool (maybe combined with https://testinfra.readthedocs.io/en/latest/).

butjar commented 5 years ago

Test TODOs:

butjar commented 5 years ago

I tested the script on ubuntu, debian and fedora. Seems to work fine. However, fedora doesn't come with python pre-installed. Maybe we could adjust the shell script to prepare all distributions.

jadinm commented 5 years ago

I tested the script on ubuntu, debian and fedora. Seems to work fine.

Working on my side as well :+1:

However, fedora doesn't come with python pre-installed. Maybe we could adjust the shell script to prepare all distributions.

The problem is that Mininet only offers a Ubuntu VM boot script. So I wouldn't do it as long as they don't support more.