Not using all potential space gained by compression

foundObjects / zram-swap

A simple zram swap service for modern systemd Linux

MIT License

309 stars 44 forks source link

Not using all potential space gained by compression #15

Closed HackerDaGreat57 closed 8 months ago

HackerDaGreat57 commented 1 year ago

I have allocated a total size of 0.1GB (~102MB) to /dev/zram0 with compression algo zstd. Compression ratio is about 1:3, which is expected and good.

But after 100MB of compressed data is inside the zram object, it switches over to disk swapping, even though it is possible to fit about 200MB more data into the zram. Do I not understand how this works or is it a bug?

foundObjects commented 1 year ago

How are you checking swap space used? Is there a chance you're checking real memory used rather than virtual?

The script isn't doing anything that would limit how much you can swap into a zram device, at least I'm pretty sure it isn't. You might try looking through your kernel virtual memory settings too, your distro may be shipping something that causes this behavior.

Just as a note I run ~24GB (8-ishGB compressed) zram swaps on all of my Proxmox boxes and I've never had an issue utilizing as much as needed.

HackerDaGreat57 commented 1 year ago

Edit: forgot to mention I changed my zram size from 102M to 6.5G.

I'm using htop's zram bar to monitor the zram usage. My current understanding of the zram bar gives me this information:

The total amount of compressed data that the zram device can store is equal to 6.50G.
There is 542M of compressed data inside the zram device as of now.
That 542M is actually equivalent to 2.44G of decompressed data.
Since we have a compression ratio of about 4.5 using zstd algo (which is an AWESOME ratio) I can approximate how much more potential memory there is to fill up: 4.5 * 6.50 ≈ 29.25G of total usable memory is available, if all of the zram device is willing to be exhausted with compressed data.

^^ I'm confused about this last part. Will Linux stop using the zram device as soon as it is storing 6.5G of compressed data? Because that frankly wouldn't make any sense compared to my understanding.

HackerDaGreat57 commented 11 months ago

So apparently you can just set a total size of x*y where x is the amount of RAM you're willing to allocate for zram. This value is multiplied by y, which is the average compression ratio you've seen for your algorithm. (This means that you have to use a ludicrous compression ratio target in your config to get the best possible results.)

If the compression craps out, though, and doesn't get the desired ratio, I'm sorry to say you're screwed. So might wanna subtract 0.5 from y before making the calculation just to be safe.