A while after joining a new (federated) room, /sync starts timing out and Element is unable to connect

PureTryOut commented 3 years ago

Background information

Dendrite version or git SHA: 0.3.4
Monolith or Polylith?: Monolith
SQLite3 or Postgres?: PostgreSQL
Running in Docker?: No
go version: 1.15.6

Description

I'm running Dendrite on a fairly low-power aarch64 machine, namely a RockPro64. Dendrite runs well and can keep up with big rooms like Matrix HQ without many problems. However, when I try to make it join any new (federated) room, an issue pops up. After it syncs such a new room for a while, it just stops responding to /sync requests.

Element reports it gets a 504 Gateway timeout result back after 120003ms (2 minutes), so I'm guessing it's just Nginx refusing to wait for a response any longer. Interestingly enough when I restart Dendrite it acts normal again, but it seems federation has been down in that time as well as rooms are out of date and have to be backfilled again. The federation tester still reports a successful result however, and CPU load is basically non-existent when this is all happening.

Steps to reproduce

Setup Dendrite on a RockPro64 or any other low powered device
Join a new room over federation
Wait a few minutes for syncs to time out while you have the newly joined room open

jaywink commented 3 years ago

Sound like #1676 I've hit a few times - though for me I don't remember joining any new rooms, but the symptoms are definitely the same.

PureTryOut commented 3 years ago

Ah yes, you do seem to be right. It might be a pure coincidence that it happens for me after joining new rooms, as it does sound very much the same. I'll close this issue as a duplicate.

matrix-org / dendrite