Dendrite crashed, failing to start now with a 'duplicate key' error.

[amackenzie]

Background information

• Dendrite version or git SHA: fdd534f86a493c26f6935cf31e721b1a2605de8c
• Monolith or Polylith?: Monolith
• SQLite3 or Postgres?: Postgres
• Running in Docker?: No
• go version: go version go1.15.6 linux/amd64

Description

I noticed that Element couldn't connect to my server, checking it revealed that Dendrite had crashed at some point and restarting it gives the following error:

I removed things that look like keys/IDs in this log, but let me know if you need those.

INFO[2021-01-20T17:37:42.972895168Z] [github.com/matrix-org/dendrite/setup/base.go:344] func2
         Starting external Monolith listener on :8008
PANI[2021-01-20T17:37:42.975462001Z] [github.com/matrix-org/dendrite/syncapi/consumers/roomserver.go:220] onOldRoomEvent
         roomserver output log: write old event failure  error="d.Topology.InsertEventInTopology: pq: duplicate key value violates unique constraint \"syncapi_output_room_events_topology_pkey\"" event="{\"auth_events\":[\"$removed\",\"$removed\",\"$removed\"],\"content\":{\"body\":\"It's a bug because the spec document is wrong, not that its the correct approach \",\"msgtype\":\"m.text\"},\"depth\":164791,\"hashes\":{\"sha256\":\"removed\"},\"origin\":\"t2l.io\",\"origin_server_ts\":removed,\"prev_events\":[\"$removed\"],\"prev_state\":[],\"room_id\":\"!removed:matrix.org\",\"sender\":\"@removed\",\"type\":\"m.room.message\",\"signatures\":{\"t2l.io\":{\"ed25519:removed\"}},\"unsigned\":{\"age\":918340}}"
panic: (*logrus.Entry) 0xc00037fdc0

goroutine 98 [running]:
github.com/sirupsen/logrus.Entry.log(0xc0000dcc40, 0xc0002a07b0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
        github.com/sirupsen/logrus@v1.7.0/entry.go:255 +0x325
github.com/sirupsen/logrus.(*Entry).Log(0xc00037fc70, 0x0, 0xc00012f9a8, 0x1, 0x1)
        github.com/sirupsen/logrus@v1.7.0/entry.go:283 +0xf0
github.com/sirupsen/logrus.(*Entry).Logf(0xc00037fc70, 0xc000000000, 0x1154d46, 0x2e, 0x0, 0x0, 0x0)
        github.com/sirupsen/logrus@v1.7.0/entry.go:329 +0xe5
github.com/sirupsen/logrus.(*Entry).Panicf(...)
        github.com/sirupsen/logrus@v1.7.0/entry.go:367
github.com/matrix-org/dendrite/syncapi/consumers.(*OutputRoomEventConsumer).onOldRoomEvent(0xc000262060, 0x12a8180, 0xc000036078, 0xc000411220, 0xc00040e1e0, 0x0)
        github.com/matrix-org/dendrite/syncapi/consumers/roomserver.go:220 +0x2d5
github.com/matrix-org/dendrite/syncapi/consumers.(*OutputRoomEventConsumer).onMessage(0xc000262060, 0xc0003e8500, 0xc400000001, 0x8400000101)
        github.com/matrix-org/dendrite/syncapi/consumers/roomserver.go:106 +0x4d8
github.com/matrix-org/dendrite/internal.(*ContinualConsumer).consumePartition(0xc00040e050, 0x12aae40, 0xc0004108e0)
        github.com/matrix-org/dendrite/internal/consumers.go:112 +0xe2
created by github.com/matrix-org/dendrite/internal.(*ContinualConsumer).StartOffsets
        github.com/matrix-org/dendrite/internal/consumers.go:102 +0x525

Steps to reproduce

• Start dendrite in monolith mode.

[pandalec]

Same here. Not able to start dendrite anymore