CPU usage bars show less usage than expected

[grahamperrin]

Screenshot below:

• GKrellM sysutils/gkrellm2 2.3.11_1 in the foreground
• htop sysutils/htop 3.2.2_1 in the mid-ground
• htop 3.3.0-dev in the background, built from source 2023-03-06.

Without bisecting, I wonder whether https://github.com/htop-dev/htop/commit/e207c8aebdcdb88bc8ab838e2ac3dd1774d6a618 (2023-03-04) produces the intended improvement in cases such as this.

Running at the time of the first shot:

cd /usr/src && time make -j4 buildworld > & buildworld.log && time make DISABLE_VULNERABILITIES=yes -j4 buildkernel > & buildkernel.log && tail buildworld.log && tail buildkernel.log

---

Later, whilst building lang/gcc12 using poudriere-bulk(8) with MAKE_JOBS enabled:

• note, the CPU row in output from top(1).

I'm fairly certain that usage of each CPU is at, or very close to, its maximum because I hear the fan(s) running fast for an extended period.

Environment

% uname -aKU
FreeBSD mowa219-gjp4-8570p-freebsd 14.0-CURRENT FreeBSD 14.0-CURRENT #33 main-n261014-cd406ac94d8b: Sun Feb 19 01:35:14 GMT 2023     grahamperrin@mowa219-gjp4-8570p-freebsd:/usr/obj/usr/src/amd64.amd64/sys/GENERIC-NODEBUG amd64 1400081 1400081
% 

[grahamperrin]

Hmm, this seems more reasonable, not long after an update to the OS:

• note, 299% kernel according to gkrelltop.

FreeBSD mowa219-gjp4-8570p-freebsd 14.0-CURRENT FreeBSD 14.0-CURRENT #34 main-n261465-22bf2a479f68: Sun Mar 12 05:18:09 GMT 2023 grahamperrin@mowa219-gjp4-8570p-freebsd:/usr/obj/usr/src/amd64.amd64/sys/GENERIC-NODEBUG amd64 1400082 1400082

[grahamperrin]

With c878343784f23d7cb18ccec6aa034f01aee8069e (2023-04-05), two screenshots from yesterday (2023-04-11):

Probably running at the time:

poudriere bulk -j main -J 8 -Ctv emulators/virtualbox-ose

– with ALLOW_MAKE_JOBS=yes in /usr/local/etc/poudriere.conf.

Translated: with this allowance, I expect use of all four CPUs to be maximised for long periods.

In the situation pictured above:

• GKrellM painted a truer picture than htop.

[cgzones]

using stress(1) on FreeBSD 13.2 seems to report usage fine

[grahamperrin]

I might try stress(1) later, thanks.

In the meantime, screenshots from 14:07 and 14:10 this afternoon (again, whilst building and packaging software):

If you visualise the four chunks of used, combined, it's nowhere near the 200% used by two processes alone.

---

A few minutes later, CPU usage percentages presented by poudriere itself, during the final ~25 seconds of packaging texlive-texmf:

[grahamperrin]

Hmm, I just realised, I forgot to remove the patch file that came from:

https://github.com/htop-dev/htop/issues/1193#issuecomment-1454474359

It's now gone, and I have rebuilt htop.

Would there have been an adverse effect from building with the patch still present?

[BenBE]

Hmm, I just realised, I forgot to remove the patch file that came from:

https://github.com/htop-dev/htop/issues/1193#issuecomment-1454474359

It's now gone, and I have rebuilt htop.

Would there have been an adverse effect from building with the patch still present?

Nothing related to CPU usage …

[grahamperrin]

htop 3.4.0-dev-3.3.0-143-g2b4feef

This appears more realistic:

[BenBE]

The large number of red (terminated) marked processes and the specifics I gather from your described work-load hints at the possibility that we're dealing with loads of short-lived processes here, thus many processes take CPU time but are never caught by htop when it scans for processes.

This could be resolved by explicit listening for short-lived processes with available kernel facilities (but that would require root and will not be available on most platforms).