search

atlanticwave-sdx / sdx-docs

Documentation for AtlanticWave-SDX project
https://www.atlanticwave-sdx.net
MIT License
0 stars 0 forks source link

Inserting openvswitch module failed #29

Open sajith opened 1 month ago

sajith commented 1 month ago

This perhaps is a question to @italovalcy rather than an issue.

I am following https://sdx-docs.readthedocs.io/en/latest/sdx_deploy_single_server.html. I did a docker compose up. Now curl -s http://0.0.0.0:8181/api/kytos/topology/v3/switches etc gets a response of {"switches":{}}.

This might be the problem:

$ docker logs mininet
modprobe: ERROR: could not insert 'openvswitch': Operation not permitted
Inserting openvswitch module ... failed!
rmmod: ERROR: ../libkmod/libkmod-module.c:856 kmod_module_remove_module() could not remove 'bridge': Operation not permitted
rmmod: ERROR: could not remove module bridge: Operation not permitted
removing bridge module ... failed!
/etc/openvswitch/conf.db does not exist ... (warning).
Creating empty database /etc/openvswitch/conf.db.
nice: cannot set niceness: Permission denied
Starting ovsdb-server.
Configuring Open vSwitch system IDs.
modprobe: ERROR: could not insert 'openvswitch': Operation not permitted
Inserting openvswitch module ... failed!
rmmod: ERROR: ../libkmod/libkmod-module.c:856 kmod_module_remove_module() could not remove 'bridge': Operation not permitted
rmmod: ERROR: could not remove module bridge: Operation not permitted
removing bridge module ... failed!

I am running this on Debian 12.7, on x86_64 hardware. I have not made any changes to the compose spec. What could be the problem?

sajith commented 1 month ago

Ok, I finally understood what is going on. I was running Docker in rootless mode, and that seems to affect mininet container's ability to load kernel modules.

https://docs.docker.com/engine/security/rootless/

I switched to Docker "rootful" mode, and that seems to solve the problem.

$ docker logs mininet 
Inserting openvswitch module.
/etc/openvswitch/conf.db does not exist ... (warning).
Creating empty database /etc/openvswitch/conf.db.
Starting ovsdb-server.
Configuring Open vSwitch system IDs.
Starting ovs-vswitchd.
Enabling remote OVSDB managers.

Now the new problem is that kytos-sdx containers keep restarting.

$ docker logs amlight
Starting enhanced syslogd: rsyslogdrsyslog startup failure, child did not respond within startup timeout (60 seconds)
 already started.
/etc/openvswitch/conf.db does not exist ... (warning).
Creating empty database /etc/openvswitch/conf.db.
Starting ovsdb-server.
Configuring Open vSwitch system IDs.
Starting ovs-vswitchd.
Enabling remote OVSDB managers.
Starting Kytos controller in a tmux session: done
tail: cannot open '/var/log/syslog' for reading: No such file or directory
tail: no files remaining

The container keeps restarting, and the above log messages keep repeating.

sajith commented 1 month ago

These lines are troublesome:

https://github.com/atlanticwave-sdx/sdx-continuous-development/blob/8b3d43fbdda9d938bd471e9c7216dce28829ad17/data-plane/docker-compose.yml#L29

https://github.com/atlanticwave-sdx/sdx-continuous-development/blob/8b3d43fbdda9d938bd471e9c7216dce28829ad17/data-plane/docker-compose.yml#L57

https://github.com/atlanticwave-sdx/sdx-continuous-development/blob/8b3d43fbdda9d938bd471e9c7216dce28829ad17/data-plane/docker-compose.yml#L85

Upon commenting them out, I was able to get things running with docker compose restart. I could also get a response out of http://0.0.0.0:{port}/api/kytos/topology/v3/switches. I also had to do docker restart tenet once, because it errored out the first time.

sajith commented 1 month ago

The trouble with rsyslogd seems to be a real one; I don't know who should handle that.

The trouble with kernel module is one of my own making. I could add these to a troubleshooting section in https://sdx-docs.readthedocs.io/en/latest/sdx_deploy_single_server.html. Assigning this issue to myself for now.