Physical memory larger than virtual memory on Windows

[BTOdell]

Describe the bug OS: Windows 10 sysinfo version: v0.30.5

When running a small Rust binary that uses sysinfo to print out process stats, I noticed that the reported virtual memory is ~5MB but the physical memory is ~15MB. According to the docs, virtual memory should always include the physical memory size and then add additional address space from paged regions, etc.

Here is what Resource Monitor shows for the process:

This is what Task Manager shows:

The "Private" memory is what Windows considers to be the uniquely allocated memory of the process:

I looked into the Windows code here and I think there is a case to be made to use a different value than WorkingSetSize for the physical memory of the process.

The question is how is this value calculated? It seems to be the "Working Set Size" minus the "Shareable" memory. Thoughts?

To Reproduce

Run a small Rust app that prints out the virtual and physical memory using sysinfo.

[BTOdell]

I tracked down what the .NET Runtime does for this API: https://learn.microsoft.com/en-us/dotnet/api/system.diagnostics.process.privatememorysize64?view=net-8.0

https://github.com/dotnet/runtime/blob/main/src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.cs#L499

https://github.com/dotnet/runtime/blob/main/src/libraries/System.Diagnostics.Process/src/System/Diagnostics/ProcessManager.Win32.cs#L366

Win32 API for accessing "Private Bytes" of process: https://learn.microsoft.com/en-us/windows/win32/api/winternl/nf-winternl-ntquerysysteminformation#system_process_information

Seems to me like this header is for internal use only and is unstable across Windows versions. This documentation claims that GetProcessMemoryInfo can also provide this information, but as it turns out only using the PROCESS_MEMORY_COUNTERS_EX2 struct which is only available on Windows 10 22H2+.

[GuillaumeGomez]

I suppose we can use the internal use only API for the time being, wouldn't be the first time for windows. Surprisingly enough, they almost never change. Interested into sending a fix?

[BTOdell]

I can take a stab at it this weekend, but I'm not super familiar with Rust yet (beginner-intermediate), so no guarantees on producing something workable.

[GuillaumeGomez]

It's fine. I can help you with the code through reviews. :)

[axetroy]

Can private memory be exposed via some api or struct?

Because Windows Task Manager counts Private Memory instead of RSS.

This results in the memory statistics being larger than the memory in Windows Task Manager, which is very confusing for common users (Not Developer).

[GuillaumeGomez]

Didn't know about Private Memory and I found a nice post about it: https://stackoverflow.com/a/1986486

So in here, I don't really know what to answer. I'd like to keep the API unified across supported OSes as much as possible, but I also want to provide the expected information.

[axetroy]

I am making a process manager. Since the memory statistics of sysinfo are RSS, and the windows task manager displays Private Bytes by default, this leads to the display difference between the two applications.

As far as I know, Private Bytes should be supported by all systems.

[GuillaumeGomez]

If you're up for implementing this feature, then I'd gladly merge it. :)

[axetroy]

I investigated the implementation of each platform.

Some of these platforms don't have a simple implementation, so forget it.

Windows:

https://microsoft.github.io/windows-docs-rs/doc/windows/System/struct.ProcessMemoryReport.html#method.PrivateWorkingSetUsage

Linux:

cat /proc/<PID>/status

https://man7.org/linux/man-pages/man5/proc_pid_status.5.html

FreeBSD

Unknown

MacOS:

Unknown

[GuillaumeGomez]

Having Linux and Windows implementation is already good enough. Just add methods returning 0 for the other platforms and I'll take a look later on.