Optimize key names for redis memory usage

[ckcr4lyf]

Currently, the key names we use are [infohash]_seeders , [infohash]_leechers and [infohash]_cache for the ZSETs.

As per BitTorrent spec, the infohash is 20 (raw) bytes, which currently we hex-encode.

So if the infohash was 20 of the byte "0x41", we would actually store it as "4141414141414141414141414141414141414141" , which takes twice as much space.

Also, instead of _seeders , _leechers and _cache , we can just do _s, _l & _c.

Old: "4141414141414141414141414141414141414141_seeders" = 48 bytes New: "\x41\x41\x41\x41\x41\x41\x41\x41\x41\x41\x41\x41\x41\x41\x41\x41\x41\x41\x41\x41_s" = 22 bytes!

[ckcr4lyf]

Some PoC:

The old ZSET for TORRENTS had ~1.7 million members, with each member being a 40 character hex string.

Old memory usage and query:

127.0.0.1:6379> MEMORY USAGE TORRENTS
(integer) 220885566
127.0.0.1:6379> ZRANGEBYSCORE TORRENTS -inf +inf WITHSCORES LIMIT 0 1
1) "51ab868dc567a523a195f9678777bd9eb233bc3e"
2) "1730944505082"

New after migration:

localhost:6363> MEMORY USAGE TORRENTS
(integer) 179512422
localhost:6363> ZRANGEBYSCORE TORRENTS -inf +inf WITHSCORES LIMIT 0 1
1) "Q\xab\x86\x8d\xc5g\xa5#\xa1\x95\xf9g\x87w\xbd\x9e\xb23\xbc>"
2) "1730944505082"

= ~39MiB of savings

Number of members:

localhost:6363> ZCARD TORRENTS
(integer) 1723881

Which times 20 bytes is equal to ~32MiB of raw saving

Migration done via: https://github.com/ckcr4lyf/kiryuu/commit/e9ae6e7ed21f9c2db2b556d2f8a4a376bd66a7fa#diff-24aefee293dfadfa2a31a8ad54a768fd5900431bc56309b46986a3a1bed78c16R7-R19