ckcr4lyf / kiryuu

A highly performant HTTP bittorrent tracker (WIP)
Do What The F*ck You Want To Public License
13 stars 3 forks source link

Improve cleanup #15

Open ckcr4lyf opened 1 year ago

ckcr4lyf commented 1 year ago

Current cleanup has a 0.1% chance per announce: https://github.com/ckcr4lyf/kiryuu/blob/9d45a1187962f20351410cb9bb81c08be76106ef/src/main.rs#L171-L176

This affects non popular torrents where the swarm dies of, e.g. in this one, the total (including stale) count in redis is 345, whereas the active (last 30 min) is just 8!

$ redis-cli
127.0.0.1:6379> ZCOUNT '4b0a62cb3d03d6e27a3878293ec9233b09254d74_seeders' 0 +inf
(integer) 345
127.0.0.1:6379> ZCOUNT '4b0a62cb3d03d6e27a3878293ec9233b09254d74_seeders' 1675089032225 1675090832225
(integer) 8

This introduces additional CPU strain per 8 announces per 30 min, vs. a more regular cleanup might be better (for less popular torrents)

ckcr4lyf commented 1 year ago

From cleaning all torrents:

image

Total -> 160/460 = ~38%!!!

ckcr4lyf commented 1 year ago

image

On first (seeder) cleanup, I killed metrics so there is a gap. But the leecher clean shows no noticeable change in announce per min, latency during the cleaning

ckcr4lyf commented 1 year ago

Cleanup is added to kouko, would be better if it can be within kiryuu (kouko is in TS/JS so even SCAN + loops are heavy on CPU)