xapi-project / xcp-networkd

The XCP networking daemon
Other
14 stars 42 forks source link

Network_interface.Internal_error("Not_found") - /proc/net/bonding/ #48

Closed bugsyb closed 9 years ago

bugsyb commented 9 years ago

For some reasons running xe pif-scan host-uuid=....

it looks for /proc/net/bonding/ whilst it does not exist on normal system. Tested on cleanly installed system too. System in question: Ubuntu 14.04 LTS, Xen 4.4 (amd64). I'm guessing it might be related to: https://github.com/xapi-project/xcp-networkd/issues/38

bridge setup in standard Linux way using brctl and properly visible at Xen level.

xe network-list uuid ( RO) : 5369a468-f639-b7ec-ddc1-b1d5861da84e name-label ( RW): Pool-wide network associated with em1 name-description ( RW): bridge ( RO): brem1

em1 is standard ethernet interface.

xe pif-list uuid ( RO) : b6a218d3-8f7b-f9cd-c1f1-3f16a63cc785 device ( RO): em1 currently-attached ( RO): true VLAN ( RO): -1 network-uuid ( RO): 5369a468-f639-b7ec-ddc1-b1d5861da84e

trying to scan for interfaces: xe pif-scan host-uuid=0d83e699-df0b-40db-9934-b5cbc9e5eafe The server failed to handle your request, due to an internal error. The given message may give details useful for debugging the problem.

Syslog messages: Jan 2 14:06:09 silver xcp-networkd: [error|silver|174|PIF.scan R:01e4a6892f20|network_utils] Error: could not read /proc/net/bonding/brem1.

Jan 2 14:06:09 silver xapi: [debug|silver|1989 UNIX /var/lib/xcp/xapi||cli] Xapi_cli.exception_handler: Got exception INTERNAL_ERROR: [ Network_interface.Internal_error("Not_found") ]

Full dump: Jan 2 14:06:09 silver xapi: [ info|silver|1989 UNIX /var/lib/xcp/xapi||cli] xe pif-scan host-uuid=0d83e699-df0b-40db-9934-b5cbc9e5eafe username=root password=(omitted) Jan 2 14:06:09 silver xapi: [ info|silver|1989 UNIX /var/lib/xcp/xapi|session.login_with_password D:a1aba3612cfd|xapi] Session.create trackid=25343f1677c75cc6c8b5dbee602036ba pool=false uname=root originator=cli is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49 Jan 2 14:06:09 silver xapi: [debug|silver|1989 UNIX /var/lib/xcp/xapi|session.login_with_password D:a1aba3612cfd|mscgen] xapi=>xapi [label="session.get_uuid"]; Jan 2 14:06:09 silver xapi: [debug|silver|1990 UNIX /var/lib/xcp/xapi||dummytaskhelper] task dispatch:session.get_uuid D:b7d7411afdb3 created by task D:a1aba3612cfd Jan 2 14:06:09 silver xapi: [debug|silver|1989 UNIX /var/lib/xcp/xapi|PIF.scan R:01e4a6892f20|audit] PIF.scan: host = '0d83e699-df0b-40db-9934-b5cbc9e5eafe (silver)' Jan 2 14:06:09 silver xcp-networkd: [error|silver|174|PIF.scan R:01e4a6892f20|network_utils] Error: could not read /proc/net/bonding/brem1. Jan 2 14:06:09 silver xapi: [debug|silver|1989 UNIX /var/lib/xcp/xapi|PIF.scan R:01e4a6892f20|backtrace] Called from file \"list.ml\", line 55, characters 20-23\nCalled from file \"xapi_pif.ml\", line 261, characters 4-68\nCalled from file \"xapi_pif.ml\", line 479, characters 9-37\nCalled from file \"message_forwarding.ml\", line 233, characters 25-44\nCalled from file \"rbac.ml\", line 227, characters 16-23\n Jan 2 14:06:09 silver xapi: 20150102T13:06:09.062Z|audit|silver|1989 UNIX /var/lib/xcp/xapi|PIF.scan R:01e4a6892f20|audit ]' 'API' 'PIF.scan' (('host' 'silver' '0d83e699-df0b-40db-9934-b5cbc9e5eafe' 'OpaqueRef:0d0d7bf7-4d03-afc4-4280-8724014224ba'))) Jan 2 14:06:09 silver xapi: [debug|silver|1989 UNIX /var/lib/xcp/xapi|PIF.scan R:01e4a6892f20|backtrace] Raised at file \"rbac.ml\", line 236, characters 10-15\nCalled from file \"server_helpers.ml\", line 75, characters 11-41\n Jan 2 14:06:09 silver xapi: [debug|silver|1989 UNIX /var/lib/xcp/xapi|PIF.scan R:01e4a6892f20|dispatcher] Server_helpers.exec exception_handler: Got exception INTERNAL_ERROR: [ Network_interface.Internal_error("Not_found") ] Jan 2 14:06:09 silver xapi: [debug|silver|1989 UNIX /var/lib/xcp/xapi|PIF.scan R:01e4a6892f20|dispatcher] Raised at file \"rbac.ml\", line 236, characters 10-15\nCalled from file \"server_helpers.ml\", line 75, characters 11-41\n Jan 2 14:06:09 silver xapi: [debug|silver|1989 UNIX /var/lib/xcp/xapi|PIF.scan R:01e4a6892f20|backtrace] Raised at file \"rbac.ml\", line 236, characters 10-15\nCalled from file \"server_helpers.ml\", line 75, characters 11-41\n Jan 2 14:06:09 silver xapi: [debug|silver|1989 UNIX /var/lib/xcp/xapi|PIF.scan R:01e4a6892f20|xapi] Raised at file \"server_helpers.ml\", line 90, characters 14-15\nCalled from file \"lib/pervasiveext.ml\", line 22, characters 2-9\n Jan 2 14:06:09 silver xapi: [debug|silver|1989 UNIX /var/lib/xcp/xapi|dispatch:PIF.scan D:a8b297444b68|xapi] Raised at file \"lib/pervasiveext.ml\", line 26, characters 22-25\nCalled from file \"lib/debug.ml\", line 57, characters 17-20\nRe-raised at file \"lib/debug.ml\", line 62, characters 10-11\nCalled from file \"lib/pervasiveext.ml\", line 22, characters 2-9\n Jan 2 14:06:09 silver xapi: [debug|silver|1989 UNIX /var/lib/xcp/xapi|dispatch:PIF.scan D:a8b297444b68|backtrace] Raised at file \"lib/pervasiveext.ml\", line 26, characters 22-25\nCalled from file \"lib/debug.ml\", line 57, characters 17-20\nRe-raised at file \"lib/debug.ml\", line 62, characters 10-11\nCalled from file \"lib/pervasiveext.ml\", line 22, characters 2-9\nRe-raised at file \"lib/pervasiveext.ml\", line 26, characters 22-25\nCalled from file \"server_helpers.ml\", line 136, characters 10-106\nCalled from file \"server.ml\", line 19252, characters 23-161\nCalled from file \"server_helpers.ml\", line 115, characters 4-7\n Jan 2 14:06:09 silver xapi: [debug|silver|1989 UNIX /var/lib/xcp/xapi||xapi] Raised at file \"client.ml\", line 6, characters 37-75\nCalled from file \"client.ml\", line 18, characters 55-110\nCalled from file \"client.ml\", line 8682, characters 6-53\nCalled from file \"cli_operations.ml\", line 3672, characters 10-45\nCalled from file \"xapi_cli.ml\", line 112, characters 18-56\nCalled from file \"lib/pervasiveext.ml\", line 22, characters 2-9\n Jan 2 14:06:09 silver xapi: [ info|silver|1989 UNIX /var/lib/xcp/xapi|session.logout D:e73c5f0ec695|xapi] Session.destroy trackid=25343f1677c75cc6c8b5dbee602036ba Jan 2 14:06:09 silver xapi: [debug|silver|1989 UNIX /var/lib/xcp/xapi||backtrace] Raised at file \"lib/pervasiveext.ml\", line 26, characters 22-25\nCalled from file \"xapi_cli.ml\", line 111, characters 2-138\nCalled from file \"xapi_cli.ml\", line 221, characters 7-44\nCalled from file \"xapi_cli.ml\", line 273, characters 4-23\n Jan 2 14:06:09 silver xapi: [debug|silver|1989 UNIX /var/lib/xcp/xapi||cli] Xapi_cli.exception_handler: Got exception INTERNAL_ERROR: [ Network_interface.Internal_error("Not_found") ] Jan 2 14:06:09 silver xapi: [debug|silver|1989 UNIX /var/lib/xcp/xapi||cli] Raised at file \"lib/pervasiveext.ml\", line 26, characters 22-25\nCalled from file \"xapi_cli.ml\", line 111, characters 2-138\nCalled from file \"xapi_cli.ml\", line 221, characters 7-44\nCalled from file \"xapi_cli.ml\", line 273, characters 4-23\n

simonjbeaumont commented 9 years ago

There is a problem with 3.12 and newer kernels. We are working on a fix but essentially (quoting @dvrabel)...

This is because xcp-networkd assumes that the presence /sys/class/net/<dev>/master implies that is a bond. But as of Linux 3.12, openvswitch ports have a master link to the bridge device.

I think xcp-networkd should check /sys/class/net/<dev>/master/bonding/slaves exists and contains before treating as a Linux bond slave.

This bug breaks XenServer installations because one of the first boot scripts does a xe pif-scan and this fails.

bugsyb commented 9 years ago

Thanks!

Good to know it's known issue, so hopefully will be fixed some time in the future.

Regards, Dawid

2015-01-07 15:37 GMT+01:00 Si Beaumont notifications@github.com:

There is a problem with 3.12 and newer kernels. We are working on a fix but essentially (quoting @dvrabel https://github.com/dvrabel)...

This is because xcp-networkd assumes that the presence /sys/class/net//master implies that is a bond. But as of Linux 3.12, openvswitch ports have a master link to the bridge device.

I think xcp-networkd should check /sys/class/net//master/bonding/slaves exists and contains before treating as a Linux bond slave.

This bug breaks XenServer installations because one of the first boot scripts does a xe pif-scan and this fails.

— Reply to this email directly or view it on GitHub https://github.com/xapi-project/xcp-networkd/issues/48#issuecomment-69029539 .

simonjbeaumont commented 9 years ago

@bugsyb said:

hopefully will be fixed some time in the future.

It's being worked on right now in fact :)