CLOSED - I'm an idiot

[kha84]

Describe the bug

See the screenshot. Make a note on what free/used memory various tools are reporting. Whenever free -m, top, htop are all showing one thing, bpytop is showing a completely off figure, like 10% or something more. Either it's because you decided to use some strange units - MiB, GiB instead of usual Megabytes and Gigabytes, or because of some bug within psutil.virtual_memory which provides all this information.

To Reproduce

Run side-by-side: 1) watch "free -m" 2) top 3) htop 4) bpytop

Expected behavior

bpytop should be showing memory figure close to the 'golden standard' free gnu tool.

Screenshots

Info bpytop version: 1.0.61 psutil version: 5.8.0 Python 3.9.2 debian 11 bullseye/stable on aarch64

Additional context

contents of ~/.config/bpytop/error.log

have some errors logged there, but it looks like they're irrelevant:

ERROR: Data collection thread failed with exception: [Errno 5] Input/output error
Traceback (most recent call last):
  File "/usr/bin/bpytop", line 2895, in _runner
    if CONFIG.draw_clock and CONFIG.update_ms != 1000: Box.draw_clock()
  File "/usr/bin/bpytop", line 1699, in draw_clock
    Draw.buffer("clock", out, z=1, now=now, once=not force, only_save=Menu.active)
  File "/usr/bin/bpytop", line 1028, in buffer
    cls.out(name)
  File "/usr/bin/bpytop", line 1042, in out
    cls.now(out)
  File "/usr/bin/bpytop", line 1001, in now
    print(*args, sep="", end="", flush=True)
OSError: [Errno 5] Input/output error

[kha84]

Just discovered that you've actually made this kind of setting ("Base 10 sizes") in btop. So I switched to it. You can reclassify this ticket as "enhancement request". Anyways, your both bpytop and btop are awesome, thanks a lot for them

[aristocratos]

@kha84 First, look up what a Mebibyte (MiB) actually is: https://www.techtarget.com/searchstorage/definition/mebibyte-MiB Cause this is probably what you mean by "usual Megabytes" since it's the standard, even on Windows which write MB but actually shows MiB.

Regarding the "Used memory", this depends on which calculation is used, there is no definitive number on how much memory is in use on a modern linux system since memory can be shared, cached, etc. Take a look at output from /proc/meminfo:

MemTotal:       16325648 kB
MemFree:          254636 kB
MemAvailable:   12800996 kB
Buffers:         1825012 kB
Cached:          8437992 kB
SwapCached:         2444 kB
Active:          6847400 kB
Inactive:        5848944 kB
Active(anon):      73288 kB
Inactive(anon):  2506348 kB
Active(file):    6774112 kB
Inactive(file):  3342596 kB
Unevictable:         144 kB
Mlocked:             144 kB
SwapTotal:       5197816 kB
SwapFree:        5187320 kB
Dirty:                36 kB
Writeback:             0 kB
AnonPages:       2429168 kB
Mapped:          2368800 kB
Shmem:            150196 kB
KReclaimable:    2771196 kB
Slab:            3054508 kB
SReclaimable:    2771196 kB
SUnreclaim:       283312 kB
KernelStack:       17232 kB
PageTables:        44820 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    13360640 kB
Committed_AS:   10298064 kB
VmallocTotal:   34359738367 kB
VmallocUsed:       85432 kB
VmallocChunk:          0 kB
Percpu:            11456 kB
HardwareCorrupted:     0 kB
AnonHugePages:         0 kB
ShmemHugePages:        0 kB
ShmemPmdMapped:        0 kB
FileHugePages:         0 kB
FilePmdMapped:         0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
Hugetlb:               0 kB
DirectMap4k:     1548604 kB
DirectMap2M:    10975232 kB
DirectMap1G:     4194304 kB

There is no label called MemUsed, you could calculate it from MemTotal - MemFree but that would be incorrect because there is MemAvailable which means some of the used memory in MemTotal - MemFree are actually available for new processes to use. Then there is Active and Inactive memory... What I mean to say is that it isn't that clear how this actually should be calculated. And if you download a couple of other system monitor programs like glances, gtop, bottom etc, I bet you'll see difference there too.

[kha84]

Yeah sorry for that. I confused myself here.

Free / top were showing figures for "available", "cache" and "total" in Mebibytes, whereas btop/bpytop was using Gigibytes for them. And given base of them is not 10, conversion from Mebibytes to Gigibytes is not as simple as visually discarding 3 digits from the right side, but actually dividing by 1024

As for free memory, yes, I do agree it's not that straightforward topic, was just hoping that the logic of it's calculation will be on par with free and top.

[kha84]

Sorry for the hassle. What appeared as a bug to me, was actually me looking at wrong things

[aristocratos]

@kha84 No problem :) It's confusing since harddrive manufacturers, ISP's and others that have something to gain in obfuscating what the unit of measurement they use actually mean also keep muddying up the definitions.

The current calculation for "used memory" in bashtop, bpytop and btop++ are all calculated by total memory - available memory since it gives the best approximation of how much memory isn't immediately available for new processes.

In bpytop: https://github.com/aristocratos/bpytop/blob/ac0f6660be0ad87b23ecb7f371a4a737e1bf017f/bpytop.py#L3288-L3296

[kha84]

Yeah that makes sense completely. It's actually already on par to how the same is calculated everywhere. It was just me being blind and not seeing that in my own screenshot.

Thanks for your time and patience. And sorry once again