NixOS / nixpkgs

Nix Packages collection & NixOS
MIT License
18.38k stars 14.33k forks source link

Zram has optional recompressionn and memory tracking features (since kernel 6.2), but this is not exposed in NixOS #338166

Open ahydronous opened 3 months ago

ahydronous commented 3 months ago

From https://docs.kernel.org/admin-guide/blockdev/zram.html

Recompression

With CONFIG_ZRAM_MULTI_COMP, zram can recompress pages using alternative (secondary) compression algorithms. The basic idea is that alternative compression algorithm can provide better compression ratio at a price of (potentially) slower compression/decompression speeds. Alternative compression algorithm can, for example, be more successful compressing huge pages (those that default algorithm failed to compress). Another application is idle pages recompression - pages that are cold and sit in the memory can be recompressed using more effective algorithm and, hence, reduce zsmalloc memory usage. With CONFIG_ZRAM_MULTI_COMP, zram supports up to 4 compression algorithms: one primary and up to 3 secondary ones. Primary zram compressor is explained in “3) Select compression algorithm”, secondary algorithms are configured using recomp_algorithm device attribute.

Memory Tracking

With CONFIG_ZRAM_MEMORY_TRACKING, user can know information of the zram block. It could be useful to catch cold or incompressible pages of the process with*pagemap.

KCynk commented 3 months ago

I can confirm this option is not exposed also on unstable nixos

[vo7@87a9f8f3b6a231:~] nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 6.6.47, NixOS, 24.11 (Vicuna), 24.11pre671089.d0e1602ddde6`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.18.5`
 - channels(root): `"nixos"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

[vo7@87a9f8f3b6a231:~] cat /sys/block/zram0/comp_algorithm
cat: /sys/block/zram0/comp_algorithm: No such file or directory
[vo7@87a9f8f3b6a231:~] cat /sys/block/zram0/max_comp_streams
cat: /sys/block/zram0/max_comp_streams: No such file or directory
[vo7@87a9f8f3b6a231:~] cat /sys/class/zram-control/hot_add
cat: /sys/class/zram-control/hot_add: No such file or directory
[vo7@87a9f8f3b6a231:~]  cat /sys/block/zramX/writeback_limit
cat: /sys/block/zramX/writeback_limit: No such file or directory
[vo7@87a9f8f3b6a231:~] uname -r
6.6.47
alyraffauf commented 3 months ago

The zramSwap module is just a wrapper around zram-generator: https://github.com/systemd/zram-generator

is it possible this is an upstream limitation? I can't find anywhere to set these options with zram-generator yet, but you can set the compression algorithm, memory limits, etc.

That said, I can't replicate @KCynk's issue in my setup. I can see max_comp_streams and other options just fine. It's just the nixpkgs module that doesn't have it as an option yet.


@aly ➜ ~ cat /sys/block/zram0/comp_algorithm 
───────┬─────────────────────────────────────────────────────────────────
       │ File: /sys/block/zram0/comp_algorithm
───────┼─────────────────────────────────────────────────────────────────
   1   │ lzo lzo-rle lz4 lz4hc 842 [zstd] 
───────┴─────────────────────────────────────────────────────────────────
@aly ➜ ~ cat /sys/block/zram0/max_comp_streams 
───────┬─────────────────────────────────────────────────────────────────
       │ File: /sys/block/zram0/max_comp_streams
───────┼─────────────────────────────────────────────────────────────────
   1   │ 12
───────┴─────────────────────────────────────────────────────────────────
@aly ➜ ~ cat /sys/block/zram0/writeback_limit  
───────┬─────────────────────────────────────────────────────────────────
       │ File: /sys/block/zram0/writeback_limit
───────┼─────────────────────────────────────────────────────────────────
   1   │ 0
───────┴─────────────────────────────────────────────────────────────────
@aly ➜ ~ nix-shell -p nix-info --run "nix-info -m"
this path will be fetched (0.01 MiB download, 0.05 MiB unpacked):
  /nix/store/bc0as359y5y067dfp7afrq1f011m9ny3-stdenv-linux
copying path '/nix/store/bc0as359y5y067dfp7afrq1f011m9ny3-stdenv-linux' from 'https://cache.nixos.org'...
 - system: `"x86_64-linux"`
 - host os: `Linux 6.10.5, NixOS, 24.05 (Uakari), 24.05.20240822.797f7dc`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.18.5`
 - nixpkgs: `/nix/store/5z81la6av4j0ckp0w1949lxiwjyyykks-source`