search

dylex / slack-libpurple

Slack module for libpurple
GNU General Public License v2.0
275 stars 38 forks source link

failure to successfully connect to a slack with 30,000 users #81

Open danhimalplanet opened 6 years ago

danhimalplanet commented 6 years ago

slack-libpurple crashes when use it to connect to the Gophers slack. Gophers slack has thousands of users.

backtrace: https://gist.github.com/5913b633afd6f67408b39d75649e5583

dylex/slack-libpurple commit id ee0f2bd

dylex commented 6 years ago

Can you get a debug log up to the crash? It looks like some http request is failing, and that error is not being handled well.

danhimalplanet commented 6 years ago

https://gist.github.com/danhimalplanet/6a6168eba295197d56b7eeab8eee7e78

dylex commented 6 years ago

Strange -- it crashes after getting a ratelimited response, and then disconnecting. I'm not sure why it's disconnecting. It has been loading users for 2 minutes by that point -- maybe something higher up timed out? The ratelimiting callback definitely fails to deal with disconnection, so would crash if something external disconnected in that time. Things we should try:

  1. Figure out if anything (in bitlbee or libpurple?) has some connection overall connection timeout that we can lengthen or otherwise what's closing the connection
  2. Tweaking the ratelimit_delay option (maybe lowering it a bit)
  3. Finally adding the header parsing necessary for using the correct ratelimit delay
  4. Fixing the crash itself, though this won't fix the underlying failure
  5. Longer term, lazy-loading of users. I think the only thing the user list is used for now is searching @ references but if we limit those to channel members maybe that's easier (though I imagine #general has all the users, so maybe that just creates another problem)
krisfremen commented 6 years ago

I'm running into the same issue if I have a few of the slacks connecting at the same time.

dylex commented 6 years ago

I've been working on a lazy loading option on the "lazy" branch. The very basics are working at this point, if anyone wants to try it on a large slack, but it's definitely still in progress. Make sure you enable the "lazy loading" account option.

renatolfc commented 4 years ago

@dylex Just tested the lazy loading option (from the history branch, commit id b7f7d7c1a8b4ffbfc9710f03e54fb2f5889ca169) to load a ~9k user Slack Workspace and it worked just fine. Without it, I'm unable to log into that workspace.