search

hansode / worklogs

0 stars 0 forks source link

nested lxc error #59

Closed hansode closed 9 years ago

hansode commented 9 years ago 
+ lxc-start -n 35 -d -l DEBUG -o /var/log/lxc/35.log
lxc-start: lxc_start.c: main: 344 The container failed to start.
lxc-start: lxc_start.c: main: 346 To get more details, run the container in foreground mode.
lxc-start: lxc_start.c: main: 348 Additional information can be obtained by setting the --logfile and --logpriority options.
hansode commented 9 years ago 
$ sudo less /var/log/lxc/35.log
      lxc-start 1432879581.538 INFO     lxc_start_ui - lxc_start.c:main:264 - using rcfile /var/lib/lxc/35/config
      lxc-start 1432879581.538 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup cpuset unknown to /var/lib/lxc 35
      lxc-start 1432879581.538 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup cpu unknown to /var/lib/lxc 35
      lxc-start 1432879581.538 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup memory unknown to /var/lib/lxc 35
      lxc-start 1432879581.538 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup devices unknown to /var/lib/lxc 35
      lxc-start 1432879581.538 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup freezer unknown to /var/lib/lxc 35
      lxc-start 1432879581.538 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup net_cls unknown to /var/lib/lxc 35
      lxc-start 1432879581.538 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup blkio unknown to /var/lib/lxc 35
      lxc-start 1432879581.538 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup perf_event unknown to /var/lib/lxc 35
      lxc-start 1432879581.538 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup hugetlb unknown to /var/lib/lxc 35
      lxc-start 1432879581.539 INFO     lxc_start - start.c:lxc_check_inherited:221 - closed inherited fd 4
      lxc-start 1432879581.541 INFO     lxc_container - lxccontainer.c:do_lxcapi_start:708 - Attempting to set proc title to [lxc monitor] /var/lib/lxc 35
      lxc-start 1432879581.541 INFO     lxc_lsm - lsm/lsm.c:lsm_init:48 - LSM security driver nop
      lxc-start 1432879581.542 DEBUG    lxc_start - start.c:setup_signal_fd:259 - sigchild handler set
      lxc-start 1432879581.542 DEBUG    lxc_console - console.c:lxc_console_peer_default:536 - no console peer
      lxc-start 1432879581.542 INFO     lxc_start - start.c:lxc_init:451 - '35' is initialized
      lxc-start 1432879581.542 INFO     lxc_start - start.c:lxc_check_inherited:221 - closed inherited fd 4
      lxc-start 1432879581.542 DEBUG    lxc_start - start.c:__lxc_start:1137 - Not dropping cap_sys_boot or watching utmp
      lxc-start 1432879581.544 INFO     lxc_monitor - monitor.c:lxc_monitor_sock_name:177 - using monitor sock name lxc/ad055575fe28ddd5//var/lib/lxc
      lxc-start 1432879581.544 ERROR    lxc_conf - conf.c:instantiate_veth:2672 - failed to attach 'vethHFFFMT' to the bridge 'kemumakikol0': Operation not permitted
      lxc-start 1432879581.560 ERROR    lxc_conf - conf.c:lxc_create_network:2955 - failed to create netdev
      lxc-start 1432879581.560 ERROR    lxc_start - start.c:lxc_spawn:914 - failed to create the network
      lxc-start 1432879581.560 ERROR    lxc_start - start.c:__lxc_start:1164 - failed to spawn '35'
      lxc-start 1432879581.560 WARN     lxc_commands - commands.c:lxc_cmd_rsp_recv:172 - command get_cgroup failed to receive response
      lxc-start 1432879581.560 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup cpuset unknown to /var/lib/lxc 35
      lxc-start 1432879581.560 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup cpu unknown to /var/lib/lxc 35
      lxc-start 1432879581.560 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup memory unknown to /var/lib/lxc 35
      lxc-start 1432879581.560 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup devices unknown to /var/lib/lxc 35
      lxc-start 1432879581.560 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup freezer unknown to /var/lib/lxc 35
      lxc-start 1432879581.560 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup net_cls unknown to /var/lib/lxc 35
      lxc-start 1432879581.560 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup blkio unknown to /var/lib/lxc 35
      lxc-start 1432879581.560 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup perf_event unknown to /var/lib/lxc 35
      lxc-start 1432879581.560 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup hugetlb unknown to /var/lib/lxc 35
      lxc-start 1432879586.565 ERROR    lxc_start_ui - lxc_start.c:main:344 - The container failed to start.
      lxc-start 1432879586.565 ERROR    lxc_start_ui - lxc_start.c:main:346 - To get more details, run the container in foreground mode.
      lxc-start 1432879586.565 ERROR    lxc_start_ui - lxc_start.c:main:348 - Additional information can be obtained by setting the --logfile and --logpriority options.
      lxc-start 1432880233.551 INFO     lxc_start_ui - lxc_start.c:main:264 - using rcfile /var/lib/lxc/35/config
      lxc-start 1432880233.552 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup cpuset unknown to /var/lib/lxc 35
      lxc-start 1432880233.552 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup cpu unknown to /var/lib/lxc 35
      lxc-start 1432880233.552 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup memory unknown to /var/lib/lxc 35
      lxc-start 1432880233.552 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup devices unknown to /var/lib/lxc 35
      lxc-start 1432880233.552 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup freezer unknown to /var/lib/lxc 35
      lxc-start 1432880233.552 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup net_cls unknown to /var/lib/lxc 35
      lxc-start 1432880233.552 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup blkio unknown to /var/lib/lxc 35
      lxc-start 1432880233.552 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup perf_event unknown to /var/lib/lxc 35
      lxc-start 1432880233.552 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup hugetlb unknown to /var/lib/lxc 35
      lxc-start 1432880233.552 INFO     lxc_start - start.c:lxc_check_inherited:221 - closed inherited fd 4
      lxc-start 1432880233.554 INFO     lxc_container - lxccontainer.c:do_lxcapi_start:708 - Attempting to set proc title to [lxc monitor] /var/lib/lxc 35
      lxc-start 1432880233.554 INFO     lxc_lsm - lsm/lsm.c:lsm_init:48 - LSM security driver nop
      lxc-start 1432880233.555 DEBUG    lxc_start - start.c:setup_signal_fd:259 - sigchild handler set
      lxc-start 1432880233.555 DEBUG    lxc_console - console.c:lxc_console_peer_default:536 - no console peer
      lxc-start 1432880233.555 INFO     lxc_start - start.c:lxc_init:451 - '35' is initialized
      lxc-start 1432880233.555 INFO     lxc_start - start.c:lxc_check_inherited:221 - closed inherited fd 4
      lxc-start 1432880233.555 DEBUG    lxc_start - start.c:__lxc_start:1137 - Not dropping cap_sys_boot or watching utmp
      lxc-start 1432880233.557 INFO     lxc_monitor - monitor.c:lxc_monitor_sock_name:177 - using monitor sock name lxc/ad055575fe28ddd5//var/lib/lxc
      lxc-start 1432880233.557 ERROR    lxc_conf - conf.c:instantiate_veth:2672 - failed to attach 'vethQCI7OL' to the bridge 'kemumakikol0': Operation not permitted
      lxc-start 1432880233.563 ERROR    lxc_conf - conf.c:lxc_create_network:2955 - failed to create netdev
      lxc-start 1432880233.563 ERROR    lxc_start - start.c:lxc_spawn:914 - failed to create the network
      lxc-start 1432880233.563 ERROR    lxc_start - start.c:__lxc_start:1164 - failed to spawn '35'
      lxc-start 1432880233.564 WARN     lxc_commands - commands.c:lxc_cmd_rsp_recv:172 - command get_cgroup failed to receive response
      lxc-start 1432880233.564 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup cpuset unknown to /var/lib/lxc 35
      lxc-start 1432880233.564 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup cpu unknown to /var/lib/lxc 35
      lxc-start 1432880233.564 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup memory unknown to /var/lib/lxc 35
      lxc-start 1432880233.564 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup devices unknown to /var/lib/lxc 35
      lxc-start 1432880233.564 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup freezer unknown to /var/lib/lxc 35
      lxc-start 1432880233.564 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup net_cls unknown to /var/lib/lxc 35
      lxc-start 1432880233.564 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup blkio unknown to /var/lib/lxc 35
      lxc-start 1432880233.564 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup perf_event unknown to /var/lib/lxc 35
      lxc-start 1432880233.564 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup hugetlb unknown to /var/lib/lxc 35
      lxc-start 1432880238.569 ERROR    lxc_start_ui - lxc_start.c:main:344 - The container failed to start.
      lxc-start 1432880238.569 ERROR    lxc_start_ui - lxc_start.c:main:346 - To get more details, run the container in foreground mode.
      lxc-start 1432880238.569 ERROR    lxc_start_ui - lxc_start.c:main:348 - Additional information can be obtained by setting the --logfile and --logpriority options.
hansode commented 9 years ago

kemumakikol0設定し忘れ。

hansode commented 9 years ago 
$ sudo ./lxc-start.sh 35
+ PATH=/bin:/usr/bin:/sbin:/usr/sbin
+ ctid=35
+ [[ -z 35 ]]
+ lxc-start -n 35 -d -l DEBUG -o /var/log/lxc/35.log
lxc-start: lxc_start.c: main: 344 The container failed to start.
lxc-start: lxc_start.c: main: 346 To get more details, run the container in foreground mode.
lxc-start: lxc_start.c: main: 348 Additional information can be obtained by setting the --logfile and --logpriority options.
+ lxc-wait -n 35 -s RUNNING

まだ駄目。

hansode commented 9 years ago 
$ brctl show
bridge name     bridge id               STP enabled     interfaces
kemumakibr0             8000.000000000000       no
kemumakikol0            8000.4252b44fc5a2       no              eth0
nestbr0         8000.000000000000       no
vboxbr0         8000.000000000000       no
$ brctl show kemumakikol0
bridge name     bridge id               STP enabled     interfaces
kemumakikol0            8000.4252b44fc5a2       no              eth0
hansode commented 9 years ago 
      lxc-start 1432879581.538 INFO     lxc_start_ui - lxc_start.c:main:264 - using rcfile /var/lib/lxc/35/config
      lxc-start 1432879581.538 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup cpuset unknown to /var/lib/lxc 35
      lxc-start 1432879581.538 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup cpu unknown to /var/lib/lxc 35
      lxc-start 1432879581.538 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup memory unknown to /var/lib/lxc 35
      lxc-start 1432879581.538 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup devices unknown to /var/lib/lxc 35
      lxc-start 1432879581.538 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup freezer unknown to /var/lib/lxc 35
      lxc-start 1432879581.538 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup net_cls unknown to /var/lib/lxc 35
      lxc-start 1432879581.538 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup blkio unknown to /var/lib/lxc 35
      lxc-start 1432879581.538 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup perf_event unknown to /var/lib/lxc 35
      lxc-start 1432879581.538 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1100 - Not attaching to cgroup hugetlb unknown to /var/lib/lxc 35
      lxc-start 1432879581.539 INFO     lxc_start - start.c:lxc_check_inherited:221 - closed inherited fd 4
      lxc-start 1432879581.541 INFO     lxc_container - lxccontainer.c:do_lxcapi_start:708 - Attempting to set proc title to [lxc monitor] /var/lib/lxc 35
      lxc-start 1432879581.541 INFO     lxc_lsm - lsm/lsm.c:lsm_init:48 - LSM security driver nop
      lxc-start 1432879581.542 DEBUG    lxc_start - start.c:setup_signal_fd:259 - sigchild handler set
      lxc-start 1432879581.542 DEBUG    lxc_console - console.c:lxc_console_peer_default:536 - no console peer
      lxc-start 1432879581.542 INFO     lxc_start - start.c:lxc_init:451 - '35' is initialized
      lxc-start 1432879581.542 INFO     lxc_start - start.c:lxc_check_inherited:221 - closed inherited fd 4
      lxc-start 1432879581.542 DEBUG    lxc_start - start.c:__lxc_start:1137 - Not dropping cap_sys_boot or watching utmp
      lxc-start 1432879581.544 INFO     lxc_monitor - monitor.c:lxc_monitor_sock_name:177 - using monitor sock name lxc/ad055575fe28ddd5//var/lib/lxc
      lxc-start 1432879581.544 ERROR    lxc_conf - conf.c:instantiate_veth:2672 - failed to attach 'vethHFFFMT' to the bridge 'kemumakikol0': Operation not permitted
      lxc-start 1432879581.560 ERROR    lxc_conf - conf.c:lxc_create_network:2955 - failed to create netdev
      lxc-start 1432879581.560 ERROR    lxc_start - start.c:lxc_spawn:914 - failed to create the network
      lxc-start 1432879581.560 ERROR    lxc_start - start.c:__lxc_start:1164 - failed to spawn '35'
      lxc-start 1432879581.560 WARN     lxc_commands - commands.c:lxc_cmd_rsp_recv:172 - command get_cgroup failed to receive response
hansode commented 9 years ago

nesting時のlxcバージョン不一致が問題？ ディストリビューションとバージョンを一致させて検証。

hansode commented 9 years ago

ディストリビューションとバージョンを一致させて検証。

一致させたら動いた。

hansode commented 9 years ago

隠れた前提条件？として、nestする場合はlxcバージョンをそろえる事と言う事のようである。

hansode commented 9 years ago 
$ sudo lxc-ls --fancy --nesting
NAME        STATE    IPV4                                              IPV6  AUTOSTART
--------------------------------------------------------------------------------------
31          RUNNING  10.0.2.2, 172.16.2.2, 172.16.255.2, 192.168.2.31  -     NO
32          RUNNING  10.0.2.2, 172.16.2.2, 172.16.255.2, 192.168.2.32  -     NO
33          RUNNING  10.0.22.200, 10.0.33.200, 192.168.2.33            -     NO
34          RUNNING  10.0.2.2, 172.16.2.2, 172.16.255.2, 192.168.2.34  -     NO
 \_ 35      RUNNING  10.0.2.2, 172.16.2.2, 172.16.255.2, 192.168.2.35  -     NO
     \_ 36  RUNNING  192.168.2.36                                      -     NO
hansode commented 9 years ago

Fedora-22に問題が潜んでいる可能性が高い。