squidowl / halloy

IRC application written in Rust
GNU General Public License v3.0
1.69k stars 66 forks source link

WHO is being rate-limited #584

Open aeadio opened 2 months ago

aeadio commented 2 months ago

Reopening #293

This appears to either be unfixed, or else a regression has occurred. Happening on the latest release, 2024.12 (MacOS).

who_poll_interval and who_retry_interval within a [servers] block seem to be ignored. Snippet from config:

[servers.znc-libera]
server = "<redacted>.ts.net"
port = 6501
use_tls = false
username = "<redacted>/libera"
password = "<redacted>"
nickname = "<redacted>"
realname = "<redacted>"
who_poll_interval = 3600
who_retry_interval = 3600

I get spammed with rate-limited messages regardless of which value is provided to those tunables. In fact, the rate limiting seems to be so bad, I can't do anything on Libera after startup -- like join additional channels or send messages.

I'm not interested in seeing who is away, so it's bizarre to me that this feature is not configurable.

andymandias commented 2 months ago

ZNC supports away-notify, so you shouldn't be seeing continued WHO spam. There will be a burst when launching Halloy, but that should be it. What version of ZNC and enabled modules are you running? I would like to reproduce this behavior.

casperstorm commented 2 months ago

Thanks for (re)opening the issue! Slightly unrelated to it, but to address:

I'm not interested in seeing who is away, so it's bizarre to me that this feature is not configurable.

If there's one thing I've learned from working on Halloy, it's that no matter how much you can configure, it's never enough 😅. People who use IRC as their daily driver in 2024 each have very specific requirements, and for a new client like ours, it's hard to nail everything on the first try. Opening feature requests is the best way to let us know something is missing!

That said, it does sound a bit strange with the bug, and not something i can reproduce. Could this be a issue on your ZNC end?

aeadio commented 2 months ago

ZNC doesn't exhibit the issue with WeeChat on Linux. I'm running ZNC version 1.9.1. I'm not running a lot of modules and I haven't changed most of the settings from the defaults.

Global modules

User modules

Server modules

I'm currently in 25 channels. I don't know if the WHO spam is just the initial scrape when Halloy starts, but I've left it up for at least 15 minutes and it's still going at that point. The client is completely unusable on Libera during this time.

Edit: FWIW the config documentation claims the default poll interval is 180 seconds, and retry is 10. But I see the WHO spam happening every couple of seconds, no matter what this value is set to.

aeadio commented 2 months ago

While figuring out how to get some debug output out of ZNC, I went ahead and launched Halloy again and the behavior seems to have disappeared. I'm not sure if it began respecting my config directives, and the WHO spam will continue in an hour (what I've currently got it set to). Or possibly if it was just the initial scan, and it's finished and cached the results.

In either case, when the WHO spam does happen, it's extremely disruptive, in a way I've never seen with another IRC client. I think this issue requires more research.

I'm happy to provide more info if you guys want to point me in the direction of what would be useful to you.

andymandias commented 2 months ago

Thanks for the ZNC info! I'll see if I can't reproduce the behavior.

One thing I do know from past experience with ZNC: if you're using multiple clients (e.g. connected with both Weechat and Halloy) without the route_replies module enabled, then you wind up with a lot of WHO spam in the server buffer. Any chance that's going on here?

aeadio commented 2 months ago

That's good to know, but I don't think that's the culprit here. I've had my Linux laptop powered down the entire time I've been testing out Halloy, so no other clients connected simultaneously.

It looks like I did get some WHO spam exactly 1 hour later. Although strangely, it's just 10 lines this time. I've got my retry limit also set to 1 hour, so I'm guessing it timed out on only some of the channels (I'm currently in 26) and hasn't retried the rest yet.

aeadio commented 2 months ago

Shortly after the 1 hour mark caused a re-spam, I got disconnected from Libera, and upon reconnect was seeing a volume of WHO spam similar to the original problem. Multiple rate limit warnings per second, and seeming to go on indefinitely. No setting have been changed.

There definitely seem to be edge cases where the WHO querying turns into a storm. During this time it seems like the config directives to limit it do effectively nothing.

It feels like this feature ought to be disabled and other IRC clients should be investigated to see how they handle WHO querying, and if they've got documented workarounds in code comments or commit messages that would be useful to understand edge cases around this feature.

casperstorm commented 2 months ago

First small step is the ability to disable WHO polling all together. I've made that possible in this PR: https://github.com/squidowl/halloy/pull/599.

marcus233 commented 1 month ago

I'm seeing this WHO rate-limit while connected to libera using irccloud as a bouncer on 2024.12 on OS X and Linux. I believe irccloud has their own bouncer implementation so I don't think it's directly related to znc.

I'm also in far fewer channels at only 6 but one has 800+ (#emacs) and another 700+ (#openbsd) users. The other 4 are between 100-300. On another libera connection via irccloud bouncer with only 2 channels (30 users, and 4 users) this issue does not happen.