search

matrix-org / synapse

Synapse: Matrix homeserver written in Python/Twisted.
https://matrix-org.github.io/synapse
Apache License 2.0
11.79k stars 2.13k forks source link

Room invite API call returns error code M_UNKNOWN #9804

Open germain-gg opened 3 years ago

germain-gg commented 3 years ago

Description

Whilst debugging vector-im/element-web#16924 I found myself having to create a lot of room to reproduce the race condition described there.

Looking throught the looks I could sporadically see the room invite API endpoint returning a 500

{
    "errcode": "M_UNKNOWN",
    "error": "Internal server error"
}

Steps to reproduce

@anoadragon453 kindly agreed to look at the logs and reported

you tried to invite someone from mozilla.org, which returned a 429 ratelimiting error over federation, and matrix.org's federation client didn't know how to handle 429 errors and has returned a 500 to you as a result.

So it looks like we'll retry a certain number of times and then just raise as a 500. Which probably isn't really the best way to handle a 429 response. I'm slightly surprised that we even hit a ratelimit if we do back off though.

Version information

richvdh commented 3 years ago

I think this is basically the same as https://github.com/matrix-org/synapse/issues/2047

anoadragon453 commented 3 years ago

Somewhat yes, in the fact that it's Synapse returning a 500 to the client when receiving unexpected error codes from the remote server.

I noticed that the invite error code handling in FederationClient was quite sane with respect to 429's specifically. If it receives one, it wraps it in a SynapseError and bubbles it up to the client:

https://github.com/matrix-org/synapse/blob/44bb881096d7ad4d730e2dc31e0094c0324e0970/synapse/federation/federation_client.py#L839-L840

However, looking in MatrixFederationClient._send_request we can see that receiving a 429 will attempt to retry a few times before eventually raising a RequestSendFailed exception, which the call stack doesn't look to handle properly.

https://github.com/matrix-org/synapse/blob/1d5f0e3529ec5acd889037c8ebcca2820ad003d5/synapse/http/matrixfederationclient.py#L533-L579