search

EionRobb / purple-discord

A libpurple/Pidgin plugin for Discord
GNU General Public License v3.0
381 stars 43 forks source link

WIP: Fetch channel history at startup #376

Open dlmarquis opened 3 years ago

dlmarquis commented 3 years ago

This PR is a bit all over the place, and that's my fault for wandering off into side projects without making a separate branch like a good boy. As it is now, this branch is in a much saner state than it was this morning, but let me know if you want me to cut it up into smaller chunks.

The biggest change is some rudimentary ratelimiting that allows us to finally query missed group chat messages at startup. Experimentally, Discord has a 5-per-5-second limit on any given endpoint group (per channel, per guild, per webhook). The only place we run into that enough to cause serious problems is history fetching, so I added a 1-second delay before it fetches the next batch of messages. I also added a 30ms delay before any http fetch towards the end of not hitting the global ratelimit. The fetch process should also now abort and resend the request if it receives a code 429 ratelimit exceeded in the response header.

To make the history fetching experience a little better, I added channel settings that allow you to override whether a single channel is treated as large or small.

I also updated some enums, and added message type and permission flag enums to make the code a little less obtuse.

Outside of that, there are a few stray fixes to errors I've made in previous PRs.

EionRobb commented 3 years ago

Love all the enums and clean up. Thank you!

Sadly the sleeps will cause too many other problems :(

dlmarquis commented 3 years ago

Okay, I think I've addressed the issues now. The sucky part is I had to defer processing the READY statement to include info from READY_SUPPLEMENTAL to get the channel size handling correct, and I'm fairly certain I've handled that poorly. I'd love to brainstorm a better way to get at this.

At this point I think it makes sense to separate out the history grabbing at start from the rest. I'll make a new pull request in a moment.

EionRobb commented 3 years ago

Okay, I think I've addressed the issues now. The sucky part is I had to defer processing the READY statement to include info from READY_SUPPLEMENTAL to get the channel size handling correct, and I'm fairly certain I've handled that poorly. I'd love to brainstorm a better way to get at this.

This also assumes that READY_SUPPLEMENTAL always gets triggered, which it's not guaranteed to

dlmarquis commented 3 years ago

This also assumes that READY_SUPPLEMENTAL always gets triggered, which it's not guaranteed to

Does it not? I checked a few other projects using the user api like DISCUM, and they all assume READY_SUPPLEMENTAL gets sent every time. Do you know what conditions cause it to not get triggered?