iam-sysop / CacheMon

Windows 10 Standby Cache Monitor and Flusher
GNU General Public License v3.0
1 stars 0 forks source link

(not bug) More Info I Found #2

Open DwarfFighterCleric opened 2 years ago

DwarfFighterCleric commented 2 years ago

Hey, great work. Haven't tried this tool yet but I have been using RamMap64.exe to GREAT results. (in my experience RamMap seems to clean a bit more than using the stand-alone EmptyStandbyList.exe app for some reason, which the additional benefit of being a mouse button press, instead of having to type command lines)

I started experimenting with it but I noticed after a few minutes my RAM was back into near 70% usage, which triggers pagefile usage.
Then, I stumbled on this tool called NTCacheSet (this version still works on Windows 10 1904 onwards...)
https://www.uwe-sieber.de/ntcacheset_e.html#SetSystemFileCacheSize

After using NTCacheSet and paying attention to the area marked with color Pink in RamMap (the Metafile cache) I fully realized what was going on:
Not only Windows 10 caches frequently used apps in memory (Memory Caching). Windows 10 also caches Filesystem files (go to a folder you know is populated with thousands of files and right-click into Properties, then watch that filesystem cache grow)
That is a completely separate Cache that Windows does NOT free up when system reaches full RAM usage (it prefers to use pagefile first, as you mention it)

The solution I found to this is using both RamMap64 (manually once a day) and NtCacheSet (it runs automatically in the background every X seconds) Filesystem Cache was the culprit for the rest of free memory being exhausted.
After 8 hours of usage, I had browser with dozens of open tabs and haven't noticed my RAM going above 55% at all times, 55 at most.
You might want to add NtCacheSet functionality to your app for even more consistent results.
These are the settings I use for NtCacheSet:
(create shortcut and insert these parameters at the end)
NtCacheSet.exe /int=15000 /min=8M /max=32M /show=1 (activates each 15 seconds, this is useful for moments where the disk is busy, it works better than when I had lower interval)

I wish there was a way to at least set a hard ceiling limit for Mem Cache on Windows, that alone would avoid all these problems, would avoid all this manual labor. Automatic memory management never worked. For now, this seems to stop the recurring cycle of death that caused pagefile spikes (unnecessary writes on disk, reduces lifespan, etc)

Inside the registry location HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\SysMain
there is a key called SvcMemHardLimitInMB
I've tried values of 256 and 64 for that key,
but haven't observed Mem Caching using a limited amount at all. I wish this worked.
I wonder if manually deleting sysmain.DLL file is an option to avoid mem caching entirely?

iam-sysop commented 2 years ago

This little app actually desperately needs to be updated -- I have a couple settings to add, one to remove, and there is an NT Service version I have been working on and use locally on two test machines. The lacking feature of the Service version is there are currently no user interface to control service settings and readouts. Some other projects have really buried me and held up progress on this utility.

As for the cache behaviors - this utility monitors the file-system caching. It will run in the system tray and provides an idea of what your memory mapping looks like. You will find it matches the "Mapped File" region of RamMap (SysInternals) in it's cache monitoring - and will flush that cache automatically based on your settings.

Application working-sets are what populates the other cache space you mention, and flushing those only leads to instant re-load of their data as they are "running applications" - the feature to flush working sets will be removed in the next update to this app.

Essentially this app would be like having RamMap run in the tray, and whenever your reach your thresholds, automatically flushes the "standby cache" or mapped-file cache. In all reality, the file caching is beneficial, but Windows is too aggressive and will eat up memory before expiring items from that cache.

I appreciate the pointers towards the other two apps -- and I will definitely research them to see if there is added benefit or something that can be added as a feature.

Thank you again!