search

twilight-rs / twilight

Powerful, flexible, and scalable ecosystem of Rust libraries for the Discord API.
https://discord.gg/twilight-rs
ISC License
656 stars 130 forks source link

Editing a guild member (changing voice-channel) hangs indefinitely #2296

Open Fyko opened 10 months ago

Fyko commented 10 months ago

Howdy folks! I'm running into this odd issue where, in certain contexts, performing a PATCH Guild Member to change their voice-channel causes the program to hang indefinitely. I cannot reproduce outside of the sample I provided -- the request goes without a hitch.

Reproducable Example

https://gist.github.com/Fyko/f1b6c53843074b4668f9d03097457fcb#file-readme-md

Logs

chuckle_interactions::commands::breakout_rooms: moving user: Id<UserMarker>(492374435274162177)
twilight_http::client: url="https://discord.com/api/v10/guilds/839188384752599071/members/492374435274162177"
twilight_http_ratelimiting::in_memory: getting bucket for path: GuildsIdMembersId(839188384752599071)
twilight_http_ratelimiting::in_memory: making new bucket for path: GuildsIdMembersId(839188384752599071)
twilight_http_ratelimiting::in_memory::bucket: starting to get next in queue path=GuildsIdMembersId(839188384752599071)
twilight_http_ratelimiting::in_memory::bucket: starting to get next in queue path=InteractionCallback(1169351050944315512)
twilight_http_ratelimiting::in_memory::bucket: starting to wait for response headers
hyper::client::pool: reuse idle connection for ("https", discord.com)
hyper::proto::h1::io: flushed 404 bytes
twilight_gateway::shard: received dispatch event_type=VOICE_STATE_UPDATE sequence=28
Stream closed EOF for production/chuckle-5bfff454fb-rmh84 (chuckle) # this happens in prod, the pod dies

Trace logs: message.txt

Tokio Console

https://github.com/twilight-rs/twilight/assets/45381083/ba443d08-f3fc-4975-9b74-a5a56231ea30

Fyko commented 10 months ago

image hmm...

edit: nevermind! i think this has to do with an ongoing outage

Erk- commented 10 months ago

I think that it is only the 500 that is related to the outage, the indefinite hang probably still happens.

laralove143 commented 10 months ago

can we reproduce this with other requests returning 5xx?

Fyko commented 10 months ago

The 500 only happened because of the Discord outage. In production, it hangs indefinitely and kube eventually kills the pod.

As mentioned in the issue and repro example, I have no problem executing the query outside of my repro example.

laralove143 commented 10 months ago

the problem is that it hangs, and it'd be useful to know if it only hangs when it gets 500 on edit guild member endpoint or on any 500s, though the latter would mean it only sometimes halts

for the edit guild member endpoint, does it always hang or has this only happened once? it may be hard to reproduce since we cant simulate an outage

Fyko commented 10 months ago

it always hangs, both in production and and when testing locally