search

lxc / lxc

LXC - Linux Containers
https://linuxcontainers.org/lxc
Other
4.63k stars 1.12k forks source link

Unable to enter network namespace of container from lxc.net.[i].script.up #4458

Open mdivyamohan opened 3 months ago

mdivyamohan commented 3 months ago

Required information

# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 21.04
Release:        21.04
Codename:       hirsute

# lxc-start --version
4.0.2

# lxc-checkconfig
Kernel configuration not found at /proc/config.gz; searching...
Kernel configuration found at /boot/config-5.11.0-25-generic
--- Namespaces ---
Namespaces: enabled
Utsname namespace: enabled
Ipc namespace: enabled
Pid namespace: enabled
User namespace: enabled
Network namespace: enabled

--- Control groups ---
Cgroups: enabled

Cgroup v1 mount points:
/sys/fs/cgroup/systemd
/sys/fs/cgroup/net_cls,net_prio
/sys/fs/cgroup/hugetlb
/sys/fs/cgroup/pids
/sys/fs/cgroup/rdma
/sys/fs/cgroup/perf_event
/sys/fs/cgroup/cpuset
/sys/fs/cgroup/cpu,cpuacct
/sys/fs/cgroup/devices
/sys/fs/cgroup/blkio
/sys/fs/cgroup/freezer
/sys/fs/cgroup/memory

Cgroup v2 mount points:
/sys/fs/cgroup/unified

Cgroup v1 clone_children flag: enabled
Cgroup device: enabled
Cgroup sched: enabled
Cgroup cpu account: enabled
Cgroup memory controller: enabled
Cgroup cpuset: enabled

--- Misc ---
Veth pair device: enabled, loaded
Macvlan: enabled, not loaded
Vlan: enabled, not loaded
Bridges: enabled, loaded
Advanced netfilter: enabled, loaded
CONFIG_NF_NAT_IPV4: missing
CONFIG_NF_NAT_IPV6: missing
CONFIG_IP_NF_TARGET_MASQUERADE: enabled, not loaded
CONFIG_IP6_NF_TARGET_MASQUERADE: enabled, not loaded
CONFIG_NETFILTER_XT_TARGET_CHECKSUM: enabled, loaded
CONFIG_NETFILTER_XT_MATCH_COMMENT: enabled, not loaded
FUSE (for use with lxcfs): enabled, not loaded

--- Checkpoint/Restore ---
checkpoint restore: enabled
CONFIG_FHANDLE: enabled
CONFIG_EVENTFD: enabled
CONFIG_EPOLL: enabled
CONFIG_UNIX_DIAG: enabled
CONFIG_INET_DIAG: enabled
CONFIG_PACKET_DIAG: enabled
CONFIG_NETLINK_DIAG: enabled
File capabilities:

Note : Before booting a new kernel, you can check its configuration
usage : CONFIG=/path/to/config /usr/bin/lxc-checkconfig

# uname -a
Linux nesc-divym 5.11.0-25-generic #27-Ubuntu SMP Fri Jul 9 23:06:29 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Issue description

I want to enter the network namespace of the container from net.script.up hook. I tried two options, both does not work.

  1. Tried using the LXC_NET_NS environment variable explained in below documentation, however this variable is not set.

LXC_[NAMESPACE IDENTIFIER]_NS: path under /proc/PID/fd/ to a file descriptor referring to the container's namespace. For each preserved namespace type there will be a separate environment variable. These environment variables will only be set if lxc.hook.version is set to 1.

  1. Tried to fetch the namespace of main pid of container by running lxc-info -n <name> command, however this command hangs when executed from hook script.

Is there any other alternate way to achieve this? End goal is to be able to run nsenter command to execute a command in container net namespace.

Steps to reproduce

Relevant section of config file:

lxc.hook.version = 1

lxc.net.0.type = veth
lxc.net.0.link = lxcbr0
lxc.net.0.flags = up
lxc.net.0.script.up = /var/lib/lxc/test/up.sh

Hook script content:

# cat /var/lib/lxc/test/up.sh
#!/bin/bash

echo "up: Hello from up script of $LXC_NAME"

NET_NAMESPACE_PATH=$LXC_NET_NS

if [ -n "$NET_NAMESPACE_PATH" ]; then
    nsenter --net="$NET_NAMESPACE_PATH" ip addr list
else
    echo "up: Network namespace path not found or not defined."
fi

echo "up: Running lxc-info -n $LXC_NAME"
echo "up: $(lxc-info -n $LXC_NAME)"
echo "up: Finished running lxc-info -n $LXC_NAME"

# pid=$(lxc-info -n $LXC_NAME  -p | awk '{print $2}')
# echo "Main pid of $LXC_NAME is $pid"
# nsenter -n -t "$pid" ip addr list

Issue-1: From container log, it is seen that LXC_NET_NS variable is empty.

lxc-start test 20240619063334.465 DEBUG    conf - conf.c:run_buffer:312 - Script exec /var/lib/lxc/test/up.sh veth lxcbr0 vetheTDF4a produced output: up: Network namespace path not found or not defined.

Issue-2: Below command hangs:

# lxc-start -n test --logfile debug.log --logpriority DEBUG

lxc-info command in hook script is running forever:

# ps -eaf | grep lxc-
root     3225310 3217722  0 09:33 pts/0    00:00:00 lxc-start -n test --logfile debug.log --logpriority DEBUG
root     3225321 3225317  0 09:33 ?        00:00:00 lxc-info -n test

Information to attach

Container debug log:

lxc-start test 20240619063334.457 INFO     start - start.c:lxc_spawn:1664 - Cloned CLONE_NEWNS
lxc-start test 20240619063334.457 INFO     start - start.c:lxc_spawn:1664 - Cloned CLONE_NEWPID
lxc-start test 20240619063334.457 INFO     start - start.c:lxc_spawn:1664 - Cloned CLONE_NEWUTS
lxc-start test 20240619063334.457 INFO     start - start.c:lxc_spawn:1664 - Cloned CLONE_NEWIPC
lxc-start test 20240619063334.457 INFO     start - start.c:lxc_spawn:1664 - Cloned CLONE_NEWNET
lxc-start test 20240619063334.457 DEBUG    start - start.c:lxc_try_preserve_namespaces:165 - Preserved mnt namespace via fd 33
lxc-start test 20240619063334.457 DEBUG    start - start.c:lxc_try_preserve_namespaces:165 - Preserved pid namespace via fd 34
lxc-start test 20240619063334.457 DEBUG    start - start.c:lxc_try_preserve_namespaces:165 - Preserved uts namespace via fd 35
lxc-start test 20240619063334.457 DEBUG    start - start.c:lxc_try_preserve_namespaces:165 - Preserved ipc namespace via fd 36
lxc-start test 20240619063334.457 DEBUG    start - start.c:lxc_try_preserve_namespaces:165 - Preserved net namespace via fd 37
lxc-start test 20240619063334.457 INFO     cgfsng - cgroups/cgfsng.c:cgfsng_setup_limits_legacy:2739 - Limits for the legacy cgroup hierarchies have been setup
lxc-start test 20240619063334.458 INFO     network - network.c:instantiate_veth:290 - Retrieved mtu 1500 from lxcbr0
lxc-start test 20240619063334.461 INFO     network - network.c:instantiate_veth:334 - Attached "vetheTDF4a" to bridge "lxcbr0"
lxc-start test 20240619063334.462 INFO     conf - conf.c:run_script_argv:342 - Executing script "/var/lib/lxc/test/up.sh" for container "test"
lxc-start test 20240619063334.465 DEBUG    conf - conf.c:run_buffer:312 - Script exec /var/lib/lxc/test/up.sh veth lxcbr0 vetheTDF4a produced output: up: Hello from up script of test

lxc-start test 20240619063334.465 DEBUG    conf - conf.c:run_buffer:312 - Script exec /var/lib/lxc/test/up.sh veth lxcbr0 vetheTDF4a produced output: up: Network namespace path not found or not defined.

lxc-start test 20240619063334.465 DEBUG    conf - conf.c:run_buffer:312 - Script exec /var/lib/lxc/test/up.sh veth lxcbr0 vetheTDF4a produced output: up: Running lxc-info -n test
corsac-s commented 3 months ago

I have the same kind of wish, although I'm using lxc-attach instead of nsenter. It seems the commands blocks when the container isn't started yet, and there is not «post start» hook.

I'd be interested in a solution as well.