matrix-org / dendrite

Dendrite is a second-generation Matrix homeserver written in Go!
https://matrix-org.github.io/dendrite/
Apache License 2.0
5.67k stars 664 forks source link

v0.6.2 spams "device list update" log message 1000s of times per second #2149

Closed timmc closed 2 years ago

timmc commented 2 years ago

Background information

Description

After upgrading my staging server from 0.5.1 to 0.6.2 my Monolith.log file quickly filled up with messages like this:

time="2022-02-06T00:12:54.307813931Z" level=info msg="Sending device list update message to []" func="github.com/matrix-org/dendrite/federationapi/consumers.(*KeyChangeConsumer).onDeviceKeyMessage" file="github.com/matrix-org/dendrite/federationapi/consumers/keychange.go:149" user_id="@timmc-stg:staging.appux.com"

In about three minutes it produced over 200,000 log lines, and was still going when I shut it down, even though I was no longer connected.

Something to address in the #2127 log reduction effort, I suppose. (There as another high-output log message but it is already ticketed and linked.)

kegsay commented 2 years ago

It shouldn't be spamming that at all, let alone sending changes to no servers.

timmc commented 2 years ago

Update: Having stopped the server and started it again, it's back to spamming. (So I turned it off again.)

Possible irregularity, no idea if this is related: When I was upgrading this staging server, I initially had Jetstream persistence going to /tmp/nats and then to another directory, then finally to its current location (having deleted the previous directories).

Please let me know if there's anything I can do to aid debugging.