search

matrix-org / synapse

Synapse: Matrix homeserver written in Python/Twisted.
https://matrix-org.github.io/synapse
Apache License 2.0
11.82k stars 2.13k forks source link

Background updates should be run on the background tasks worker #12931

Open turt2live opened 2 years ago

turt2live commented 2 years ago

Synapse can support a background tasks worker for running various internal tasks over replication, however background updates (database migrations) are still handled on the main process. It seems sensible, if by name alone, to also have background updates happen on the background tasks worker to alleviate load from the main process in the event of a large migration.

The most recent background updates I can think of aren't necessarily intensive, but it's quite possible that in the future Synapse will have a need to run a CPU or memory intensive background update across the database. Such an update might be similar to the recent-ish stream_ordering column type fix where billions of rows needed to be migrated.

clokep commented 2 years ago

This should generally be safe since Synapse cannot depend on background updates being completed by at a particular time (so are by definition somewhat safe for workers?). No idea if that holds up in practice though!

turt2live commented 2 years ago

Currently you can only have 1 background task worker as well, so should be just as safe as running them on the main process.