Option to group running processes for Linux Containers

[graysky2]

Does htop allow for this? The closest I found in the man page is CTID but that applies to OpenVZ, not lxcs.

[deric]

There's a column CGROUP that will show part of container's name. Although the space very limited, htop shows only name like docker/cd3 which does not have to be unique. When you sort by this column, you'll get all processes inside a container together.

[Harvie]

the CGROUP tends to be very long string and it breaks htop display on my 1280x800px screen. Having way to show just LXC name or single number would be really great...

This is pretty useless in my opinion:

[jb-boin]

Here is a simple patch that i made to only display the LXC container name (if the pathname is /lxc/*) or display an empty value if the process is not running on a specific CGroup (pathname / or /user/*). (the pathname part of the CGroup is located after the second : character of each lines on /proc/$pid/cgroup)

If the process does not match those cases it will use the old behavior.

--- linux/LinuxProcessList.c    2018-04-11 02:13:10.000000000 +0200
+++ linux/LinuxProcessList.c    2018-05-29 16:39:06.000026159 +0200
@@ -531,6 +531,26 @@
       if (!ok) break;
       char* group = strchr(buffer, ':');
       if (!group) break;
+
+      // Each line have 3 columns separated by a ':', the 3rd column contains the pathname
+      char* groupPathname = strchr(group + 1, ':') + 1;
+      if(groupPathname) {
+         fclose(file);
+         free(process->cgroup);
+
+         if(String_startsWith(groupPathname, "/lxc/")) {
+            // The process is inside a LXC containter, for a better readability, only the container name is returned
+            process->cgroup = xStrdup(String_trim(groupPathname + 5));
+         } else if(String_startsWith(groupPathname, "/user/") || (groupPathname[0] == '/' && strlen(String_trim(groupPathname)) == 1)) {
+            // The process is not in a particular CGroup, for a better readability, an empty value is returned
+            process->cgroup = xStrdup("");
+         } else {
+            // Returning the content of the 3rd column as is
+            process->cgroup = xStrdup(String_trim(groupPathname));
+         }
+         return;
+      }
+
       if (at != output) {
          *at = ';';
          at++;

Tested and working on htop 2.2.0 on Debian Stretch.

Adding a specific LXC container field that would be different from the CGroup field would be even better i think.

[graysky2]

I built 2.2.0 with your patch but am confused... you're saying that the path of the executable needs to be /lxc/foo/bar in order to work? My distro installs containers to /var/lib/lxc ... not sure what the default is

[jb-boin]

No, it is the path of the CGroup which is the 3rd column of each lines of /proc/$pid/cgroup. Its explained in details on the man of cgroups(7) : http://man7.org/linux/man-pages/man7/cgroups.7.html

For example, this PID (11864) is running on the LXC container called 6562 :

# cat /proc/11864/cgroup 
10:freezer:/lxc/6562
9:pids:/lxc/6562
8:net_cls,net_prio:/lxc/6562
7:perf_event:/lxc/6562
6:devices:/lxc/6562
5:blkio:/lxc/6562
4:cpu,cpuacct:/lxc/6562
3:cpuset:/lxc/6562
2:memory:/lxc/6562
1:name=systemd:/lxc/6562

In this case, without my patch, htop will display 1:name=systemd:/lxc/6562 as the CGroup value. With my patch it will display 6562.

[kartoffelheinz]

jb-boin: Thank you very much for your patch, it works flawlessly on latest Debian 9 and makes our lives a little easier. Is there a reason it has not been merged with master yet?

[jb-boin]

Ideally there should be a specific column for the LXC name as the CGroup might be used for other purposes than a LXC container and there still can be pagination problems with this patch as there could be non-LXC process with a long CGroup value overflowing the column size.

It shouldnt be too hard to do it but it would be better to check before with the project maintener what would be the best course of action here as the CGroup column as is isnt of much use.

[benyanke]

Any luck on this? I'd love to see cgroup (specifically docker) support.

[joelmaxuel]

+1 for this, or a "roll-your-own" placeholder column that can include an arbitrary ps command (e.g. ps -o cgroup $pid | cut -d'/' -f 3) in the conf file (the latter would generalize for additional platforms at the cost of additional upfront work for the end user).

[maxried]

+1, this is really annoying

[jb-boin]

A new version of the patch that adds a new LXC column, based on Debian Buster version 2.2.0 :

--- htop-2.2.0.orig/linux/LinuxProcess.h    2018-04-11 02:13:10.000000000 +0200
+++ htop-2.2.0/linux/LinuxProcess.h 2020-02-26 17:55:12.829572348 +0100
@@ -70,6 +70,7 @@
    #endif
    #ifdef HAVE_CGROUP
    CGROUP = 113,
+   LXC = 119,
    #endif
    OOM = 114,
    IO_PRIORITY = 115,
@@ -78,7 +79,7 @@
    PERCENT_IO_DELAY = 117,
    PERCENT_SWAP_DELAY = 118,
    #endif
-   LAST_PROCESSFIELD = 119,
+   LAST_PROCESSFIELD = 120,
 } LinuxProcessField;

 #include "IOPriority.h"
@@ -120,6 +121,7 @@
    #endif
    #ifdef HAVE_CGROUP
    char* cgroup;
+   char* lxc;
    #endif
    unsigned int oom;
    char* ttyDevice;

--- htop-2.2.0.orig/linux/LinuxProcessList.c    2018-04-11 02:13:10.000000000 +0200
+++ htop-2.2.0/linux/LinuxProcessList.c 2020-02-26 18:16:54.986784956 +0100
@@ -531,6 +531,24 @@
       if (!ok) break;
       char* group = strchr(buffer, ':');
       if (!group) break;
+
+      if (!process->lxc) {
+         // Each line have 3 columns separated by a ':', the 3rd column contains the pathname
+         char* groupPathname = strchr(group + 1, ':') + 1;
+         if (groupPathname) {
+            if (String_startsWith(groupPathname, "/lxc/")) {
+               // The process is inside a LXC containter, for a better readability, only the container name is kept
+               char *slashpostion = strchr((groupPathname + 5), '/');
+               if (slashpostion) {
+                  // There is a '/' in the CGroup string (after the initial "/lxc/"), a '\0' will truncate the string at this position
+                  groupPathname[(slashpostion - groupPathname)] = '\0';
+               }
+               free(process->lxc);
+               process->lxc = xStrdup(String_trim(groupPathname + 5));
+            }
+         }
+      }
+
       if (at != output) {
          *at = ';';
          at++;
@@ -540,6 +558,12 @@
       left -= wrote;
    }
    fclose(file);
+   if (!process->lxc) {
+      // The process is not in a LXC container
+      free(process->lxc);
+      // To show an empty value instead of (null)
+      process->lxc = xStrdup("");
+   }
    free(process->cgroup);
    process->cgroup = xStrdup(output);
 }

--- htop-2.2.0.orig/linux/LinuxProcess.c    2020-02-26 18:19:15.000000000 +0100
+++ htop-2.2.0/linux/LinuxProcess.c 2020-02-26 18:01:28.736822325 +0100
@@ -78,6 +78,7 @@
    #endif
    #ifdef HAVE_CGROUP
    CGROUP = 113,
+   LXC = 119,
    #endif
    OOM = 114,
    IO_PRIORITY = 115,
@@ -86,7 +87,7 @@
    PERCENT_IO_DELAY = 117,
    PERCENT_SWAP_DELAY = 118,
    #endif
-   LAST_PROCESSFIELD = 119,
+   LAST_PROCESSFIELD = 120,
 } LinuxProcessField;

 #include "IOPriority.h"
@@ -128,6 +129,7 @@
    #endif
    #ifdef HAVE_CGROUP
    char* cgroup;
+   char* lxc;
    #endif
    unsigned int oom;
    char* ttyDevice;
@@ -227,6 +229,7 @@
 #endif
 #ifdef HAVE_CGROUP
    [CGROUP] = { .name = "CGROUP", .title = "    CGROUP ", .description = "Which cgroup the process is in", .flags = PROCESS_FLAG_LINUX_CGROUP, },
+   [LXC] = { .name = "LXC", .title = "       LXC ", .description = "Which LXC constainer the process is in", .flags = PROCESS_FLAG_LINUX_CGROUP, },
 #endif
    [OOM] = { .name = "OOM", .title = "    OOM ", .description = "OOM (Out-of-Memory) killer score", .flags = PROCESS_FLAG_LINUX_OOM, },
    [IO_PRIORITY] = { .name = "IO_PRIORITY", .title = "IO ", .description = "I/O priority", .flags = PROCESS_FLAG_LINUX_IOPRIO, },
@@ -274,6 +277,7 @@
    Process_done((Process*)cast);
 #ifdef HAVE_CGROUP
    free(this->cgroup);
+   free(this->lxc);
 #endif
    free(this->ttyDevice);
    free(this);
@@ -370,6 +374,7 @@
    #endif
    #ifdef HAVE_CGROUP
    case CGROUP: xSnprintf(buffer, n, "%-10s ", lp->cgroup); break;
+   case LXC:    xSnprintf(buffer, n, "%-10s ", lp->lxc); break;
    #endif
    case OOM: xSnprintf(buffer, n, Process_pidFormat, lp->oom); break;
    case IO_PRIORITY: {
@@ -453,6 +458,8 @@
    #ifdef HAVE_CGROUP
    case CGROUP:
       return strcmp(p1->cgroup ? p1->cgroup : "", p2->cgroup ? p2->cgroup : "");
+   case LXC:
+      return strcmp(p1->lxc ? p1->lxc : "", p2->lxc ? p2->lxc : "");
    #endif
    case OOM:
       return (p2->oom - p1->oom);

And a compiled package for Debian Buster : http://jbboin.phpnet.org/htop/

[graysky2]

@jb-boin - Thank you for taking a pass at a patch. I think @hishamhm might be more receptive if you forked the repo and sent a PR rather than posting a patch?

[ser]

On Ubuntu 20.04 with LXD from snap, processes in containers have cgroups like this:

12:blkio:/lxc.payload.cios,11:hugetlb:/lxc.payload.cios,10:perf_event:/lxc.payload.cios,9:devices:/lxc.payload.cios/system.slice/systemd-networkd.service,8:freezer:/lxc.payload.cios,7:net_cls,net_prio:/lxc.payload.cios,6:memory:/lxc.payload.cios/system.slice/systemd-networkd.service,5:rdma:/lxc.payload.cios,4:cpuset:/lxc.payload.cios,3:pids:/lxc.payload.cios/system.slice/systemd-networkd.service,2:cpu,cpuacct:/lxc.payload.cios,1:name=systemd:/lxc.payload.cios/system.slice/systemd-networkd.service,0::/lxc.payload.cios/system.slice/systemd-networkd.service

[jb-boin]

On Ubuntu 20.04 with LXD from snap, processes in containers have cgroups like this:

12:blkio:/lxc.payload.cios,11:hugetlb:/lxc.payload.cios,10:perf_event:/lxc.payload.cios,9:devices:/lxc.payload.cios/system.slice/systemd-networkd.service,8:freezer:/lxc.payload.cios,7:net_cls,net_prio:/lxc.payload.cios,6:memory:/lxc.payload.cios/system.slice/systemd-networkd.service,5:rdma:/lxc.payload.cios,4:cpuset:/lxc.payload.cios,3:pids:/lxc.payload.cios/system.slice/systemd-networkd.service,2:cpu,cpuacct:/lxc.payload.cios,1:name=systemd:/lxc.payload.cios/system.slice/systemd-networkd.service,0::/lxc.payload.cios/system.slice/systemd-networkd.service

In this case, the container name is "cios"?

[ser]

Indeed.

[jb-boin]

What you pasted does not look like what i have on my test Ubuntu 20.04 LTS which doesn't use , as a separator but newlines (a process inside a LXC container created using LXD with the lxc command) :

12:freezer:/lxc.payload.buster
11:memory:/lxc.payload.buster/system.slice/networking.service
10:devices:/lxc.payload.buster/system.slice/networking.service
9:rdma:/lxc.payload.buster
8:hugetlb:/lxc.payload.buster
7:blkio:/lxc.payload.buster
6:cpu,cpuacct:/lxc.payload.buster
5:perf_event:/lxc.payload.buster
4:net_cls,net_prio:/lxc.payload.buster
3:cpuset:/lxc.payload.buster
2:pids:/lxc.payload.buster/system.slice/networking.service
1:name=systemd:/lxc.payload.buster/system.slice/networking.service
0::/lxc.payload.buster/system.slice/networking.service

So i modified the previous patch to also work with /lxc.payload. but it would not truncate what is after the , if /proc/$pid/cgroup is like what you pasted :

--- htop-2.2.0.orig/linux/LinuxProcess.h    2018-04-11 02:13:10.000000000 +0200
+++ htop-2.2.0/linux/LinuxProcess.h 2020-02-26 17:55:12.829572348 +0100
@@ -70,6 +70,7 @@
    #endif
    #ifdef HAVE_CGROUP
    CGROUP = 113,
+   LXC = 119,
    #endif
    OOM = 114,
    IO_PRIORITY = 115,
@@ -78,7 +79,7 @@
    PERCENT_IO_DELAY = 117,
    PERCENT_SWAP_DELAY = 118,
    #endif
-   LAST_PROCESSFIELD = 119,
+   LAST_PROCESSFIELD = 120,
 } LinuxProcessField;

 #include "IOPriority.h"
@@ -120,6 +121,7 @@
    #endif
    #ifdef HAVE_CGROUP
    char* cgroup;
+   char* lxc;
    #endif
    unsigned int oom;
    char* ttyDevice;

--- htop-2.2.0.orig/linux/LinuxProcessList.c    2018-04-11 02:13:10.000000000 +0200
+++ htop-2.2.0/linux/LinuxProcessList.c 2020-07-06 17:41:57.036834737 +0200
@@ -531,6 +531,33 @@
       if (!ok) break;
       char* group = strchr(buffer, ':');
       if (!group) break;
+
+      if (!process->lxc) {
+         // Each line have 3 columns separated by a ':', the 3rd column contains the pathname
+         char* groupPathname = strchr(group + 1, ':') + 1;
+         if (groupPathname) {
+            if (String_startsWith(groupPathname, "/lxc/")) {
+               // The process is inside a LXC container using CGroup V1, for a better readability, only the container name is kept
+               char *slashpostion = strchr((groupPathname + 5), '/');
+               if (slashpostion) {
+                  // There is a '/' in the CGroup string (after the initial "/lxc/"), a '\0' will truncate the string at this position
+                  groupPathname[(slashpostion - groupPathname)] = '\0';
+               }
+               free(process->lxc);
+               process->lxc = xStrdup(String_trim(groupPathname + 5));
+            } else if (String_startsWith(groupPathname, "/lxc.payload.")) {
+               // The process is inside a LXC container using CGroup V2, for a better readability, only the container name is kept
+               char *slashpostion = strchr((groupPathname + 13), '/');
+               if (slashpostion) {
+                  // There is a '/' in the CGroup string (after the initial "/lxc.payload."), a '\0' will truncate the string at this position
+                  groupPathname[(slashpostion - groupPathname)] = '\0';
+               }
+               free(process->lxc);
+               process->lxc = xStrdup(String_trim(groupPathname + 13));
+            }
+         }
+      }
+
       if (at != output) {
          *at = ';';
          at++;
@@ -540,6 +567,12 @@
       left -= wrote;
    }
    fclose(file);
+   if (!process->lxc) {
+      // The process is not in a LXC container
+      free(process->lxc);
+      // To show an empty value instead of (null)
+      process->lxc = xStrdup("");
+   }
    free(process->cgroup);
    process->cgroup = xStrdup(output);
 }

--- htop-2.2.0.orig/linux/LinuxProcess.c    2020-02-26 18:19:15.000000000 +0100
+++ htop-2.2.0/linux/LinuxProcess.c 2020-02-26 18:01:28.736822325 +0100
@@ -78,6 +78,7 @@
    #endif
    #ifdef HAVE_CGROUP
    CGROUP = 113,
+   LXC = 119,
    #endif
    OOM = 114,
    IO_PRIORITY = 115,
@@ -86,7 +87,7 @@
    PERCENT_IO_DELAY = 117,
    PERCENT_SWAP_DELAY = 118,
    #endif
-   LAST_PROCESSFIELD = 119,
+   LAST_PROCESSFIELD = 120,
 } LinuxProcessField;

 #include "IOPriority.h"
@@ -128,6 +129,7 @@
    #endif
    #ifdef HAVE_CGROUP
    char* cgroup;
+   char* lxc;
    #endif
    unsigned int oom;
    char* ttyDevice;
@@ -227,6 +229,7 @@
 #endif
 #ifdef HAVE_CGROUP
    [CGROUP] = { .name = "CGROUP", .title = "    CGROUP ", .description = "Which cgroup the process is in", .flags = PROCESS_FLAG_LINUX_CGROUP, },
+   [LXC] = { .name = "LXC", .title = "       LXC ", .description = "Which LXC constainer the process is in", .flags = PROCESS_FLAG_LINUX_CGROUP, },
 #endif
    [OOM] = { .name = "OOM", .title = "    OOM ", .description = "OOM (Out-of-Memory) killer score", .flags = PROCESS_FLAG_LINUX_OOM, },
    [IO_PRIORITY] = { .name = "IO_PRIORITY", .title = "IO ", .description = "I/O priority", .flags = PROCESS_FLAG_LINUX_IOPRIO, },
@@ -274,6 +277,7 @@
    Process_done((Process*)cast);
 #ifdef HAVE_CGROUP
    free(this->cgroup);
+   free(this->lxc);
 #endif
    free(this->ttyDevice);
    free(this);
@@ -370,6 +374,7 @@
    #endif
    #ifdef HAVE_CGROUP
    case CGROUP: xSnprintf(buffer, n, "%-10s ", lp->cgroup); break;
+   case LXC:    xSnprintf(buffer, n, "%-10s ", lp->lxc); break;
    #endif
    case OOM: xSnprintf(buffer, n, Process_pidFormat, lp->oom); break;
    case IO_PRIORITY: {
@@ -453,6 +458,8 @@
    #ifdef HAVE_CGROUP
    case CGROUP:
       return strcmp(p1->cgroup ? p1->cgroup : "", p2->cgroup ? p2->cgroup : "");
+   case LXC:
+      return strcmp(p1->lxc ? p1->lxc : "", p2->lxc ? p2->lxc : "");
    #endif
    case OOM:
       return (p2->oom - p1->oom);

I re-compiled the Debian package on a Debian 10 and tested it both on Debian 10 and Ubuntu 20.04 : It seems to be working as expected on both.

The test package is available at http://jbboin.phpnet.org/htop/

[ser]

It works fantastic, @jb-boin , it would be cool to have it in the mainline package.

[graysky2]

@jb-boin - Thank you for taking a pass at a patch. I think @hishamhm might be more receptive if you forked the repo and sent a PR rather than posting a patch?

Can you fork and send @hishamhm a PR?

[graysky2]

@jb-boin I compiled with your patch on Arch Linux, when I start DISPLAY=:0 xeyes in my lxc, I don't see any different output in htop run on the host system. For example:

8811  facade  20  0 9216  4592  4192  S 0.0 0.0 0:00.42 xeyes
9214  facade  20  0 2992  4992  4948  S 0.0 0.0 0:00.30 xscreensaver -no-splash

Is that to be expected?

[ser]

is it lxc or lxd, i have tested on lxd and it works 100% properly, of course you need to add a column in config (F2)

[graysky2]

@ser - lxc... I see now that I had to add the column. It works as expected. @jb-boin - My only suggestion is that you change the justification on the column from what looks like a right-justified to a left-justified.

[ser]

in my htop it is left justified and i would prefer right justified LOL, IMHO it should be justified the same as PID is