Oleksii-Melnyk-UA
commented
2 years ago Issue is fixed - should add -v /usr:/host/usr:ro , then the module is built correctly. https://docs.sysdig.com/en/docs/installation/sysdig-agent/agent-installation/agent-install-non-orchestrated/.
After build of new image with 3033.2.2 Flatcar version, and run of new AWS EC2 instance - sysdig-agent docker container fails to start. We use the latest version of sysdig-agent image - https://hub.docker.com/r/sysdig/agent/tags?page=1 All our other containers are running successfully on 3033.2.2 version. Also tested that latest version of sysdig-agent is running successfully on older 2905.2.6 Flatcar version.
Environment and steps to reproduce Set-up: Flatcar Linux 3033.2.2 Task: Run sysdig-agent docker container Action(s): In systemd there is a command how to start sysdig-agent docker container
ExecStart=/usr/bin/docker -H unix:///var/run/docker.sock run --name=sysdig-agent --privileged --net host --pid host --log-driver none -v /var/run/docker.sock:/host/var/run/docker.sock -v /dev:/host/dev -v /proc:/host/proc:ro -v /boot:/host/boot:ro -v /lib/modules:/host/lib/modules:ro -v /etc/sysdig/dragent.yaml:/opt/draios/etc/dragent.yaml sysdig/agent:latest
After start or restart of service, receive errors systemctl restart service sysdig.service
Logs: Feb 22 11:26:21 worker-blue-12-208 systemd[1]: sysdig.service: Scheduled restart job, restart counter is at 215. Feb 22 11:26:21 worker-blue-12-208 systemd[1]: Stopped SysDig Cloud Agent. Feb 22 11:26:21 worker-blue-12-208 systemd[1]: Starting SysDig Cloud Agent... Feb 22 11:26:21 worker-blue-12-208 docker[952591]: sysdig-agent Feb 22 11:26:21 worker-blue-12-208 docker[952602]: sysdig-agent Feb 22 11:26:21 worker-blue-12-208 docker[952613]: Using default tag: latest Feb 22 11:26:21 worker-blue-12-208 docker[952613]: latest: Pulling from sysdig/agent Feb 22 11:26:21 worker-blue-12-208 docker[952613]: Digest: sha256:217c755c4ee5ff2795491c45a3f46dd0f087d954eb6866c29cd0e0cb2c69f8cb Feb 22 11:26:21 worker-blue-12-208 docker[952613]: Status: Image is up to date for sysdig/agent:latest Feb 22 11:26:21 worker-blue-12-208 docker[952613]: docker.io/sysdig/agent:latest Feb 22 11:26:21 worker-blue-12-208 systemd[1]: Started SysDig Cloud Agent. Feb 22 11:26:21 worker-blue-12-208 docker[952631]: Setting up /usr/src links from host Feb 22 11:26:21 worker-blue-12-208 docker[952631]: ls: cannot access /host/usr/src: No such file or directory Feb 22 11:26:21 worker-blue-12-208 docker[952631]: Setting up kernel tools Feb 22 11:26:21 worker-blue-12-208 docker[952631]: Found original host ld: /lib64/ld-linux-x86-64.so.2 Feb 22 11:26:21 worker-blue-12-208 docker[952631]: Resolved host ld: /host/usr/lib64/ld-linux-x86-64.so.2 Feb 22 11:26:21 worker-blue-12-208 docker[952631]: Couldn't find host dl interpreter (or already patched) Feb 22 11:26:21 worker-blue-12-208 docker[952631]: Will use /usr/bin/gcc-9.3 to build kernel module if needed Feb 22 11:26:21 worker-blue-12-208 docker[952631]: Found preconfigured dragent.yaml Feb 22 11:26:21 worker-blue-12-208 docker[952631]: Mounting memory cgroup fs (using subsys memory) Feb 22 11:26:21 worker-blue-12-208 docker[952631]: mount: permission denied Feb 22 11:26:21 worker-blue-12-208 docker[952631]: Mounting cpu cgroup fs (using subsys cpu) Feb 22 11:26:21 worker-blue-12-208 docker[952631]: mount: permission denied Feb 22 11:26:21 worker-blue-12-208 docker[952631]: Mounting cpuacct cgroup fs (using subsys cpuacct) Feb 22 11:26:21 worker-blue-12-208 docker[952631]: mount: permission denied Feb 22 11:26:21 worker-blue-12-208 docker[952631]: Mounting cpuset cgroup fs (using subsys cpuset) Feb 22 11:26:21 worker-blue-12-208 docker[952631]: mount: permission denied Feb 22 11:26:21 worker-blue-12-208 docker[952631]: Mounting freezer cgroup fs (using subsys freezer) Feb 22 11:26:21 worker-blue-12-208 docker[952631]: mount: permission denied Feb 22 11:26:21 worker-blue-12-208 docker[952631]: failed to mount freezer cgroup subsystem Feb 22 11:26:21 worker-blue-12-208 docker[952631]: Loading kernel probe Feb 22 11:26:21 worker-blue-12-208 docker[952631]: Unloading sysdigcloud-probe, if present Feb 22 11:26:21 worker-blue-12-208 docker[952631]: Evaluating override of environment variables Feb 22 11:26:21 worker-blue-12-208 docker[952631]: Looking for kernel configuration Feb 22 11:26:21 worker-blue-12-208 docker[952631]: Found kernel config at /proc/config.gz Feb 22 11:26:21 worker-blue-12-208 docker[952631]: Running dkms install for draios-agent Feb 22 11:26:22 worker-blue-12-208 docker[952631]: Building module: Feb 22 11:26:22 worker-blue-12-208 docker[952631]: cleaning build area... Feb 22 11:26:25 worker-blue-12-208 docker[952631]: make -j8 KERNELRELEASE=5.10.96-flatcar -C /lib/modules/5.10.96-flatcar/build M=/var/lib/dkms/draios-agent/12.3.0/build...(bad exit status: 2) Feb 22 11:26:25 worker-blue-12-208 docker[952631]: Error! Bad return status for module build on kernel: 5.10.96-flatcar (x86_64) Feb 22 11:26:25 worker-blue-12-208 docker[952631]: Consult /var/lib/dkms/draios-agent/12.3.0/build/make.log for more information. Feb 22 11:26:25 worker-blue-12-208 docker[952631]: Running dkms build failed, dumping /var/lib/dkms/draios-agent/12.3.0/build/make.log Feb 22 11:26:25 worker-blue-12-208 docker[952631]: DKMS make.log for draios-agent-12.3.0 for kernel 5.10.96-flatcar (x86_64) Feb 22 11:26:25 worker-blue-12-208 docker[952631]: Tue Feb 22 11:26:22 UTC 2022 Feb 22 11:26:25 worker-blue-12-208 docker[952631]: make: Entering directory /host/lib/modules/5.10.96-flatcar/build' Feb 22 11:26:25 worker-blue-12-208 docker[952631]: CC [M] /var/lib/dkms/draios-agent/12.3.0/build/main.o Feb 22 11:26:25 worker-blue-12-208 docker[952631]: CC [M] /var/lib/dkms/draios-agent/12.3.0/build/dynamic_params_table.o Feb 22 11:26:25 worker-blue-12-208 docker[952631]: CC [M] /var/lib/dkms/draios-agent/12.3.0/build/fillers_table.o Feb 22 11:26:25 worker-blue-12-208 docker[952631]: CC [M] /var/lib/dkms/draios-agent/12.3.0/build/flags_table.o Feb 22 11:26:25 worker-blue-12-208 docker[952631]: CC [M] /var/lib/dkms/draios-agent/12.3.0/build/ppm_events.o Feb 22 11:26:25 worker-blue-12-208 docker[952631]: CC [M] /var/lib/dkms/draios-agent/12.3.0/build/ppm_fillers.o Feb 22 11:26:25 worker-blue-12-208 docker[952631]: CC [M] /var/lib/dkms/draios-agent/12.3.0/build/event_table.o Feb 22 11:26:25 worker-blue-12-208 docker[952631]: CC [M] /var/lib/dkms/draios-agent/12.3.0/build/syscall_table.o Feb 22 11:26:25 worker-blue-12-208 docker[952631]: scripts/basic/fixdep: /lib64/libc.so.6: version GLIBC_2.33' not found (required by scripts/basic/fixdep) Feb 22 11:26:25 worker-blue-12-208 docker[952631]: make[2]: ** [/var/lib/dkms/draios-agent/12.3.0/build/dynamic_params_table.o] Error 1 Feb 22 11:26:25 worker-blue-12-208 docker[952631]: make[2]: Deleting file /var/lib/dkms/draios-agent/12.3.0/build/dynamic_params_table.o' Feb 22 11:26:25 worker-blue-12-208 docker[952631]: make[2]: Waiting for unfinished jobs.... Feb 22 11:26:25 worker-blue-12-208 docker[952631]: scripts/basic/fixdep: /lib64/libc.so.6: version GLIBC_2.33' not found (required by scripts/basic/fixdep) Feb 22 11:26:25 worker-blue-12-208 docker[952631]: make[2]: [/var/lib/dkms/draios-agent/12.3.0/build/flags_table.o] Error 1 Feb 22 11:26:25 worker-blue-12-208 docker[952631]: make[2]: Deleting file /var/lib/dkms/draios-agent/12.3.0/build/flags_table.o' Feb 22 11:26:25 worker-blue-12-208 docker[952631]: scripts/basic/fixdep: /lib64/libc.so.6: version GLIBC_2.33' not found (required by scripts/basic/fixdep) Feb 22 11:26:25 worker-blue-12-208 docker[952631]: make[2]: [/var/lib/dkms/draios-agent/12.3.0/build/event_table.o] Error 1 Feb 22 11:26:25 worker-blue-12-208 docker[952631]: make[2]: Deleting file /var/lib/dkms/draios-agent/12.3.0/build/event_table.o' Feb 22 11:26:25 worker-blue-12-208 docker[952631]: scripts/basic/fixdep: /lib64/libc.so.6: version GLIBC_2.33' not found (required by scripts/basic/fixdep) Feb 22 11:26:25 worker-blue-12-208 docker[952631]: make[2]: [/var/lib/dkms/draios-agent/12.3.0/build/fillers_table.o] Error 1 Feb 22 11:26:25 worker-blue-12-208 docker[952631]: make[2]: Deleting file /var/lib/dkms/draios-agent/12.3.0/build/fillers_table.o' Feb 22 11:26:25 worker-blue-12-208 docker[952631]: scripts/basic/fixdep: /lib64/libc.so.6: version GLIBC_2.33' not found (required by scripts/basic/fixdep) Feb 22 11:26:25 worker-blue-12-208 docker[952631]: make[2]: [/var/lib/dkms/draios-agent/12.3.0/build/syscall_table.o] Error 1 Feb 22 11:26:25 worker-blue-12-208 docker[952631]: make[2]: Deleting file /var/lib/dkms/draios-agent/12.3.0/build/syscall_table.o' Feb 22 11:26:25 worker-blue-12-208 docker[952631]: scripts/basic/fixdep: /lib64/libc.so.6: version GLIBC_2.33' not found (required by scripts/basic/fixdep) Feb 22 11:26:25 worker-blue-12-208 docker[952631]: make[2]: [/var/lib/dkms/draios-agent/12.3.0/build/ppm_events.o] Error 1 Feb 22 11:26:25 worker-blue-12-208 docker[952631]: make[2]: Deleting file /var/lib/dkms/draios-agent/12.3.0/build/ppm_events.o' Feb 22 11:26:25 worker-blue-12-208 docker[952631]: scripts/basic/fixdep: /lib64/libc.so.6: version GLIBC_2.33' not found (required by scripts/basic/fixdep) Feb 22 11:26:25 worker-blue-12-208 docker[952631]: make[2]: [/var/lib/dkms/draios-agent/12.3.0/build/main.o] Error 1 Feb 22 11:26:25 worker-blue-12-208 docker[952631]: make[2]: Deleting file /var/lib/dkms/draios-agent/12.3.0/build/main.o' Feb 22 11:26:25 worker-blue-12-208 docker[952631]: scripts/basic/fixdep: /lib64/libc.so.6: version GLIBC_2.33' not found (required by scripts/basic/fixdep) Feb 22 11:26:25 worker-blue-12-208 docker[952631]: make[2]: [/var/lib/dkms/draios-agent/12.3.0/build/ppm_fillers.o] Error 1 Feb 22 11:26:25 worker-blue-12-208 docker[952631]: make[2]: Deleting file /var/lib/dkms/draios-agent/12.3.0/build/ppm_fillers.o' Feb 22 11:26:25 worker-blue-12-208 docker[952631]: make[1]: [/var/lib/dkms/draios-agent/12.3.0/build] Error 2 Feb 22 11:26:25 worker-blue-12-208 docker[952631]: make: ** [__sub-make] Error 2 Feb 22 11:26:25 worker-blue-12-208 docker[952631]: make: Leaving directory /host/lib/modules/5.10.96-flatcar/build' Feb 22 11:26:25 worker-blue-12-208 docker[952631]: Trying to load a system sysdigcloud-probe, if present Feb 22 11:26:25 worker-blue-12-208 docker[952631]: Trying to find precompiled sysdigcloud-probe for 5.10.96-flatcar Feb 22 11:26:25 worker-blue-12-208 docker[952631]: Trying to find precompiled sysdigcloud-probe for 5.10.96-flatcar Feb 22 11:26:25 worker-blue-12-208 docker[952631]: * Trying to download precompiled module from https://download.sysdig.com/stable/sysdig-probe-binaries/sysdigcloud-probe-12.3.0-x86_64-5.10.96-flatcar-940e960481d770672750b5bde0493304.ko Feb 22 11:26:25 worker-blue-12-208 docker[952631]: Download of sysdigcloud-probe for version 12.3.0 failed. Feb 22 11:26:25 worker-blue-12-208 docker[952631]: The probe for this version does not exist in the repo. Feb 22 11:26:25 worker-blue-12-208 docker[952631]: The Sysdig Agent kernel probe could not be built and a probe could not be found to download. Upgrading to the latest version of the Sysdig agent could solve this problem. For additional as> Feb 22 11:26:25 worker-blue-12-208 docker[952631]: Cannot load the probe Feb 22 11:26:25 worker-blue-12-208 systemd[1]: sysdig.service: Main process exited, code=exited, status=1/FAILURE Feb 22 11:26:25 worker-blue-12-208 docker[953486]: sysdig-agent Feb 22 11:26:25 worker-blue-12-208 systemd[1]: sysdig.service: Failed with result 'exit-code'. Feb 22 11:26:52 worker-blue-12-208 systemd[1]: Stopped SysDig Cloud Agent.
Expected behavior sysdig-agent docker container successfully started.