search

hashicorp / nomad-driver-lxc

HashiCorp Nomad LXC driver plugin
Mozilla Public License 2.0
31 stars 19 forks source link

very noisy task log #17

Closed onlyjob closed 5 years ago

onlyjob commented 5 years ago

I'm running example job on template = "/usr/share/lxc/templates/lxc-debian" and the following is logged more than once per second to example-lxc.log in the alloc directory:

lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.219 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_basecg_debuginfo:1035 - kernel subsystem 0: cpuset
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.219 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_basecg_debuginfo:1035 - kernel subsystem 1: rdma
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.219 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_basecg_debuginfo:1035 - kernel subsystem 2: net_cls
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.219 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_basecg_debuginfo:1035 - kernel subsystem 3: net_prio
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.219 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_basecg_debuginfo:1035 - kernel subsystem 4: memory
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.219 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_basecg_debuginfo:1035 - kernel subsystem 5: blkio
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.219 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_basecg_debuginfo:1035 - kernel subsystem 6: devices
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.219 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_basecg_debuginfo:1035 - kernel subsystem 7: pids
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.219 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_basecg_debuginfo:1035 - kernel subsystem 8: freezer
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.219 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_basecg_debuginfo:1035 - kernel subsystem 9: perf_event
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.219 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_basecg_debuginfo:1035 - kernel subsystem 10: cpu
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.219 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_basecg_debuginfo:1035 - kernel subsystem 11: cpuacct
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.219 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_basecg_debuginfo:1035 - kernel subsystem 12: cgroup2
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.219 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_basecg_debuginfo:1038 - named subsystem 0: name=systemd
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.219 TRACE    cgfsng - cgroups/cgfsng.c:cg_hybrid_init:2431 - No controllers are enabled for delegation in the unified hierarchy
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgfsng - cgroups/cgfsng.c:cg_hybrid_init:2459 - Writable cgroup hierarchies:
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_hierarchies:1012 -   Hierarchies:
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_hierarchies:1017 -   0: base_cgroup: /
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_hierarchies:1018 -       mountpoint:  /sys/fs/cgroup/unified
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_hierarchies:1019 -       controllers:
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_hierarchies:1017 -   1: base_cgroup: /
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_hierarchies:1018 -       mountpoint:  /sys/fs/cgroup/systemd
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_hierarchies:1019 -       controllers:
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_hierarchies:1021 -       0: name=systemd
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_hierarchies:1017 -   2: base_cgroup: /
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_hierarchies:1018 -       mountpoint:  /sys/fs/cgroup/cpu,cpuacct
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_hierarchies:1019 -       controllers:
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_hierarchies:1021 -       0: cpu
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_hierarchies:1021 -       1: cpuacct
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_hierarchies:1017 -   3: base_cgroup: /
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_hierarchies:1018 -       mountpoint:  /sys/fs/cgroup/perf_event
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_hierarchies:1019 -       controllers:
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_hierarchies:1021 -       0: perf_event
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_hierarchies:1017 -   4: base_cgroup: /
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_hierarchies:1018 -       mountpoint:  /sys/fs/cgroup/freezer
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_hierarchies:1019 -       controllers:
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_hierarchies:1021 -       0: freezer
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_hierarchies:1017 -   5: base_cgroup: /
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_hierarchies:1018 -       mountpoint:  /sys/fs/cgroup/pids
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_hierarchies:1019 -       controllers:
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_hierarchies:1021 -       0: pids
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_hierarchies:1017 -   6: base_cgroup: /
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_hierarchies:1018 -       mountpoint:  /sys/fs/cgroup/devices
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_hierarchies:1019 -       controllers:
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_hierarchies:1021 -       0: devices
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_hierarchies:1017 -   7: base_cgroup: /
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_hierarchies:1018 -       mountpoint:  /sys/fs/cgroup/blkio
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_hierarchies:1019 -       controllers:
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_hierarchies:1021 -       0: blkio
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_hierarchies:1017 -   8: base_cgroup: /
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_hierarchies:1018 -       mountpoint:  /sys/fs/cgroup/memory
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_hierarchies:1019 -       controllers:
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_hierarchies:1021 -       0: memory
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_hierarchies:1017 -   9: base_cgroup: /
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_hierarchies:1018 -       mountpoint:  /sys/fs/cgroup/net_cls,net_prio
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_hierarchies:1019 -       controllers:
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_hierarchies:1021 -       0: net_cls
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_hierarchies:1021 -       1: net_prio
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_hierarchies:1017 -   10: base_cgroup: /
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_hierarchies:1018 -       mountpoint:  /sys/fs/cgroup/rdma
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_hierarchies:1019 -       controllers:
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_hierarchies:1021 -       0: rdma
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_hierarchies:1017 -   11: base_cgroup: /
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_hierarchies:1018 -       mountpoint:  /sys/fs/cgroup/cpuset
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_hierarchies:1019 -       controllers:
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_hierarchies:1021 -       0: cpuset
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgroup - cgroups/cgroup.c:cgroup_init:56 - Initialized cgroup driver cgfsng
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgroup - cgroups/cgroup.c:cgroup_init:61 - Running with hybrid cgroup layout
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    commands - commands.c:lxc_cmd_rsp_recv:139 - Command "get_cgroup" received response
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    commands - commands.c:lxc_cmd_rsp_recv:139 - Command "get_state" received response
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 DEBUG    commands - commands.c:lxc_cmd_rsp_recv:165 - Response data length for command "get_state" is 0
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 DEBUG    commands - commands.c:lxc_cmd_get_state:585 - Container "example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8" is in "RUNNING" state
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_basecg_debuginfo:1031 - basecginfo is:
lxc example-709ef17d-99f8-08b2-a6cd-7fef22ffb5c8 20191104235625.220 TRACE    cgfsng - cgroups/cgfsng.c:lxc_cgfsng_print_basecg_debuginfo:1032 - 11:cpuset:/
10:rdma:/
9:net_cls,net_prio:/
8:memory:/init.scope
7:blkio:/init.scope
6:devices:/init.scope
5:pids:/init.scope
4:freezer:/
3:perf_event:/
2:cpu,cpuacct:/init.scope
1:name=systemd:/init.scope
0::/init.scope

Changing to template = "/usr/share/lxc/templates/lxc-busybox" makes no difference...

tgross commented 5 years ago

Hello @onlyjob and thanks for submitting this report!

That is quite a bit of logging! If you take a look at lxc.nomad#L10-L11, you'll find the logging is set to "trace" and the "verbose" flag is set. You may want to change those to turn down the noise.

onlyjob commented 5 years ago

Yes indeed, logging is much more adequate with log_level = "info".

Thanks for the tip, @tgross. Perhaps example could be corrected? log_level = "trace" is neither a good default nor a right setting for the first job to run on lxc driver.

onlyjob commented 5 years ago

Closing in favour of https://github.com/hashicorp/nomad-education-content/issues/1

Thanks.