macOS 11/Apple silicon showing high virtual memory usage

[sophiajt]

Was noticing on my mbp with an M1 Pro, the virtual memory usage is being listed as really high numbers. Nushell uses libproc get query this info, and all the virtual memory numbers that come back are hundreds of gigs.

Is there something we should be doing on the nushell side or is this libproc-related?

[andrewdavidmackenzie]

Can you share the nushell command you use to get that? I suspect its ps but would be good to know for sure.

Can you also let me know the exact macos version.

And lastly, which Resource Usage field leads to that virtual column? (Looking at nu-shell source it looks like TaskAllInfo.ptinfo.pti_virtual_size)

[andrewdavidmackenzie]

On Intel Mac using ps I get:

With all the numbers pretty high (and suspiciously similar), so I suspect wrong.

[andrewdavidmackenzie]

I notice that there are many new versions of xnu, and in the latest there is now a new V5: https://opensource.apple.com/source/xnu/xnu-7195.81.3/bsd/sys/resource.h.auto.html

but only adds a flags field, and all looks backwards compatible.

[andrewdavidmackenzie]

So, on Intel Mac, macos 12.3.1 it looks wrong also: 34.95GB

[andrewdavidmackenzie]

@akhramov Do you think this code ever returned the correct value for pti_virtual_size ?

[akhramov]

@andrewdavidmackenzie I think so.

Since you were able to reproduce the problem locally, can you try #69 and see if I guessed the root cause correctly?

[akhramov]

Apparently #69 does not fix the issue. pti_virtual_size is still ridiculously large.

[andrewdavidmackenzie]

I havent been able to confirm, but will now.

Bindgen to generate bindings makes sense.

But I doubted it would solve this problem as I checked our manual rust struct against the C source and it all matched.

Maybe need to try some test in C or ObjC to see if it works at all?

[akhramov]

Here's the repro. It doesn't work

#include <errno.h>
#include <stdio.h>
#include <string.h>
#include <unistd.h>

#include <libproc.h>

int main() {
    pid_t pid = getpid();
    struct proc_taskallinfo result = {0};

    if (errno != 0) {
        fprintf(stderr, "getpid() failed: %s \n", strerror(errno));
        return 1;
    }

    proc_pidinfo(pid, PROC_PIDTASKALLINFO, 0, &result, sizeof(result));
    if (errno != 0) {
        fprintf(stderr, "proc_pidinfo() failed: %s \n", strerror(errno));
        return 1;
    }

    printf("pti_virtual_size is %llu \n", result.ptinfo.pti_virtual_size);
    return 0;
}

I suspect it's a recent regression: https://github.com/apple/darwin-xnu/blame/2ff845c2e033bd0ff64b5b6aa6063a1f8f65aa32/osfmk/kern/bsd_kern.c#L971

[andrewdavidmackenzie]

Thanks for the repro.

Can't be that recent if showing up on Macos 11 thru 12.3 no?

[akhramov]

Probably not. There's nothing special within vm_map_adjusted_size that could have caused the issue.

https://github.com/apple/darwin-xnu/blob/main/osfmk/vm/vm_map.c#L1368-L1409

Anecdotally, others have the same problem without knowing it. Take a look at the screenshot: https://github.com/htop-dev/htop/issues/368#issuecomment-741708141

[andrewdavidmackenzie]

OK. So, while I would still like to understand this and since when it's been like this etc, it looks @jntrnr like this is a Darwin / libproc bug and not libproc-rs.

I don't see how I can fix it, but I'm fine with leaving the issue open for discussion and more info though.

[andrewdavidmackenzie]

Maybe it's our understanding of that value and what it means that's wrong? Maybe the value is correct? There is not a lot of docs on this stuff... what about BSD/Mach I wonder...

[akhramov]

Follow up:

It seems like apple's top has the same issue.

top -pid 1644 -l 1 | head
Processes: 371 total, 2 running, 369 sleeping, 2673 threads 
2022/04/17 22:22:25
Load Avg: 2.67, 2.09, 2.25 
CPU usage: 3.97% user, 28.51% sys, 67.50% idle 
SharedLibs: 471M resident, 85M data, 69M linkedit.
MemRegions: 318206 total, 1870M resident, 125M private, 3076M shared.
PhysMem: 15G used (1674M wired), 117M unused.
VM: 145T vsize, 3780M framework vsize, 134359214(0) swapins, 135892021(0) swapouts.
Networks: packets: 228108979/171G in, 118708226/88G out.
Disks: 107966461/3774G read, 47093838/3438G written.

145T for the notes app (pid 1644)? Seems too much.

[sophiajt]

I've filed a bug report with Apple (though unfortunately they don't give a link to get the status of the bug report).

With luck, maybe it'll get fixed in a future release.

[andrewdavidmackenzie]

OK, that's the best we can do then I guess.

If you get a link, you can always add here for reference.

I'll close this issue now

[daverigby]

Follow up:

It seems like apple's top has the same issue.

top -pid 1644 -l 1 | head
Processes: 371 total, 2 running, 369 sleeping, 2673 threads 
2022/04/17 22:22:25
Load Avg: 2.67, 2.09, 2.25 
CPU usage: 3.97% user, 28.51% sys, 67.50% idle 
SharedLibs: 471M resident, 85M data, 69M linkedit.
MemRegions: 318206 total, 1870M resident, 125M private, 3076M shared.
PhysMem: 15G used (1674M wired), 117M unused.
VM: 145T vsize, 3780M framework vsize, 134359214(0) swapins, 135892021(0) swapouts.
Networks: packets: 228108979/171G in, 118708226/88G out.
Disks: 107966461/3774G read, 47093838/3438G written.

145T for the notes app (pid 1644)? Seems too much.

As an aside, I don't believe this is showing the VM for the specified process; that's the sum of all OS processes - I see the same number when not filtering top by pid.

However I do see also large value for vsize for the Notes app:

$ ps -p $(pgrep Notes) -c -o comm,vsize,rss
COMM       VSZ    RSS
Notes 410800144 100736

(arm64 mac, macOS 12.4)

Interestingly the same process shows contradictory info in Activity Monitor - the summary page looks reasonable, but the details shows the same value as ps:

[akhramov]

That's... interesting. I wonder what API Apple uses.

They could use task_info, but that doesn't work unless you are root. https://github.com/apple-oss-distributions/top/blob/a989bd5d18246e330e5feadd80958b913351f8ae/libtop.c#L1572-L1577

Activity Monitor is linked against these libraries which sound suspicious:

/usr/lib/libsysmon.dylib (compatibility version 1.0.0, current version 1.0.0)
/usr/lib/libsystemstats.dylib (compatibility version 1.0.0, current version 488.0.0)

Of course, there's no headers :)

Unfortunately, my only Mac is a corporate Mac, so I can't really investigate further. Heck, even dtracing seems to require disabling SIP nowadays :)

[andrewdavidmackenzie]

If they are part of Darwin, that code is released open source and we could look there what they are doing....