Closed Pestdoktor closed 3 years ago
Also happens in !LhdcUQKgKZCiDUQRGK:matrix.org
time="2020-10-23T12:39:36.629348898Z" level=info msg="Fetched 80 events locally" func="github.com/matrix-org/dendrite/syncapi/routing.(*messagesReq).retrieveEvents" file="github.com/matrix-org/dendrite/syncapi/routing/messages.go:212" end="{{ []}}" req.id=SCK5zWiNXznF req.method=GET req.path="/_matrix/client/r0/rooms/!LhdcUQKgKZCiDUQRGK:matrix.org/messages" start="{{ []}}" user_id="@pestdoktor:fentker.eu"
time="2020-10-23T12:39:36.774378172Z" level=info msg="ServersAtEvent including 709 current events from history visibility" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).ServersAtEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:434"
time="2020-10-23T12:39:40.168951950Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$z_1jg9CQVGh9EdwVAuATpcfoyE8ZNSXxkPrmyjgK780"
time="2020-10-23T12:39:40.429255507Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$-LTavdWw8JyDyuuiHVcxO74fxxTPE4MAdnAZgrWxHIs"
time="2020-10-23T12:39:44.090964123Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$rdDcAv8FeqoWG6j8oo_6mo1ETxAv6goP_SMajaFrEEU"
time="2020-10-23T12:39:46.588596221Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$Afg1MtWaw8LyqKINupSC76ipkKmzLqix9Jv32xia8r4"
time="2020-10-23T12:39:48.114872857Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$whATK3eVJhNQ4C5QG4Db3mkJ-y6miQt_IjO39XjVC-s"
time="2020-10-23T12:39:48.675890206Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$m0XLsPs35ngSiPItFe7sHz5nmaWJRUXMUj1x7QvPXzU"
time="2020-10-23T12:39:49.313117555Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$uBi8uBeVWENTOwuXTi4uFdas1zipRMDw9gIQaH7Im_4"
time="2020-10-23T12:39:49.673487780Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$CPXhvPibaRiY8c0ehaiswIPLIolxZI8BlOcgusRqA7I"
time="2020-10-23T12:39:49.916487746Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$EVdoPRpPmQL4p9hnLsQwCiIB_Qmxb5Ec9m_pC39yZ-w"
time="2020-10-23T12:39:53.659242218Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$GquTlg1Ax6DoiOmZ2rAUlHvN46y4StwhzfjwTCbV7A8"
time="2020-10-23T12:39:53.926247721Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$ztsZ_V4iW_C0_e7mkOPe8y5jXxrWfTd7W6h_rWqnta8"
time="2020-10-23T12:39:57.911651817Z" level=info msg="Producing to topic 'DendriteOutputRoomEvent'" func="github.com/matrix-org/dendrite/roomserver/internal/input.(*Inputer).WriteOutputEvents" file="github.com/matrix-org/dendrite/roomserver/internal/input/input.go:98" adds_state=0 event_id="$1603456791309022SsKdY:matrix.org" event_type=m.room.message removes_state=0 room_id="!VtCBTOqqrBixAvWosB:matrix.org" send_as_server="" sender="@freenode_leon-p:matrix.org" type=new_room_event
time="2020-10-23T12:40:00.254369253Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$2DlYcwx3VyeV4AqeEoYV5f2UcEGJeNLaxW8GocY7y4I"
time="2020-10-23T12:40:01.799340106Z" level=info msg="Producing to topic 'DendriteOutputRoomEvent'" func="github.com/matrix-org/dendrite/roomserver/internal/input.(*Inputer).WriteOutputEvents" file="github.com/matrix-org/dendrite/roomserver/internal/input/input.go:98" adds_state=0 event_id="$1603456801309053VbhuG:matrix.org" event_type=m.room.message removes_state=0 room_id="!VtCBTOqqrBixAvWosB:matrix.org" send_as_server="" sender="@freenode_kennylevinsen:matrix.org" type=new_room_event
time="2020-10-23T12:40:01.941298396Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$c5S6KDiHVU4XCAAjAkgWnsv9NPZP_VQNgm0U2ZOVhSw"
time="2020-10-23T12:40:05.149122922Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$gK_7GUS35vD7GuGoD5B-yn61ZvD2eOlEUCZaSTsYW38"
time="2020-10-23T12:40:07.550825720Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$sRWiAatqjM5m874MPsafgdMuMjPJKhf1lR_16khb7xU"
time="2020-10-23T12:40:24.681918854Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$IUeXpQcCHtM3iWsy0V29JHsHwd3nv-uY5DE8ziUgaZs"
time="2020-10-23T12:40:25.252958588Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$Lz65HLwPgJfHWSnGQU0e8_NTOcBf8-RAV3ui3OeOJbc"
time="2020-10-23T12:40:25.681257448Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$2QfKaK_LZPjEpLgoz_TkwtcGUluGE-FdhR57jZ8vwhE"
time="2020-10-23T12:40:25.919200966Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$ssGpMqhql-kEe87nVGJIyooblMiureDxYWxpjI5wLhA"
time="2020-10-23T12:40:26.144389805Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$oDGicLB6Vr8e3bLL5P-1Sc-KhdTT6UtvWrR_P12DnwU"
time="2020-10-23T12:40:26.779313203Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$qiBqgMbrqmOg1tiOzFSq1xMrmejMe9K7GM-gJoCpNXQ"
time="2020-10-23T12:40:27.513209593Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$kSCSyyFrT51MiZldNwLBmqFcwziPv_V0prboarVP8Pw"
time="2020-10-23T12:40:27.798050421Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$ZNSQfBcP3VJhr4_nBXREzHdnHzdo5_sFUiMOF6DUCFk"
time="2020-10-23T12:40:36.220784639Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$Sewm0sM7FhKe0-u6E-N9jA2gV-qji3aFM1vwpgBG2yo"
time="2020-10-23T12:40:36.688622189Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$mebp9V6UDJmSavxXF7g6tqY10pGQ4X2SQb6X__xF0FQ"
time="2020-10-23T12:40:38.205090982Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$v42HDRDYNs06N4YpwV4_XSZeS6RkZCf-U0JlwkKeNnE"
time="2020-10-23T12:40:39.838215093Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$J06gpHwlfaWxn1h-y7h2m9hNO-oICFMRG81kKAKAqZw"
time="2020-10-23T12:40:44.674344160Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$FU4Ng-JdNW7f1RA2ohYoDrN4uyRqP8VdbGGU8BMKmOo"
time="2020-10-23T12:40:45.753190321Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$M9Rn9usXlQAxfGOt1kkLnZh2yG9Pf-MvSJ4IHs5LpRc"
time="2020-10-23T12:40:46.033925296Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$YMAYoxPOgtW6VzjCxJvNOUutCidtWiX9pWlSIxXOSDY"
time="2020-10-23T12:40:46.281142331Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$0rUcnBCb4YBuj3fSnU4gMusvwTOceqO5FolKdNjG6qo"
time="2020-10-23T12:40:50.053012912Z" level=info msg="Producing to topic 'DendriteOutputRoomEvent'" func="github.com/matrix-org/dendrite/roomserver/internal/input.(*Inputer).WriteOutputEvents" file="github.com/matrix-org/dendrite/roomserver/internal/input/input.go:98" adds_state=0 event_id="$1603456840309197bnxwr:matrix.org" event_type=m.room.message removes_state=0 room_id="!VtCBTOqqrBixAvWosB:matrix.org" send_as_server="" sender="@freenode_th0rne:matrix.org" type=new_room_event
time="2020-10-23T12:40:51.303626527Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$hV7YvR8KcGeDBNdvdbCQq6qdwdVltDLKV7nqPZuWYfg"
time="2020-10-23T12:40:54.950615346Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$K_H-W77uCTfm1uuHYEN8p6bsX_Ved_7UClNnMV1gNkA"
time="2020-10-23T12:40:55.226149309Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$vVswZHHSs8Xd3fPmw7zeDCH48rqIvgitq32jF7MRW-g"
time="2020-10-23T12:40:57.525468970Z" level=info msg="Fetched 983/983 events from the database" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:354"
time="2020-10-23T12:40:57.574066329Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$Nhm_GRTrtIZ5woGuc1ooiqgA3EmZTr5CdCT1YLq2_h4"
time="2020-10-23T12:40:59.686355806Z" level=info msg="Fetching 1 key(s)" func="github.com/matrix-org/dendrite/signingkeyserver/internal.(*ServerKeyAPI).handleFetcherKeys" file="github.com/matrix-org/dendrite/signingkeyserver/internal/api.go:208" fetcher_name="perspective server matrix.org"
time="2020-10-23T12:41:00.019069043Z" level=info msg="Updated 1 of 1 key(s) in database (0 keys remaining)" func="github.com/matrix-org/dendrite/signingkeyserver/internal.(*ServerKeyAPI).handleFetcherKeys" file="github.com/matrix-org/dendrite/signingkeyserver/internal/api.go:262" fetcher_name="perspective server matrix.org"
time="2020-10-23T12:41:04.497966417Z" level=info msg="Fetched 984/984 events from the database" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:354"
time="2020-10-23T12:41:04.698754538Z" level=info msg="Fetched 985/985 events from the database" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:354"
time="2020-10-23T12:41:04.887978433Z" level=info msg="Fetched 985/985 events from the database" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:354"
time="2020-10-23T12:41:05.189499393Z" level=info msg="Fetched 992/992 events from the database" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:354"
time="2020-10-23T12:41:05.746280297Z" level=info msg="Fetched 996/996 events from the database" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:354"
time="2020-10-23T12:41:06.217360443Z" level=info msg="Fetched 983/983 events from the database" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:354"
time="2020-10-23T12:41:06.426516301Z" level=info msg="Fetched 983/983 events from the database" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:354"
time="2020-10-23T12:41:06.653692180Z" level=info msg="Fetched 983/983 events from the database" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:354"
time="2020-10-23T12:41:07.730960020Z" level=info msg="Fetched 984/984 events from the database" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:354"
time="2020-10-23T12:41:07.936735595Z" level=info msg="Fetched 985/985 events from the database" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:354"
time="2020-10-23T12:41:08.128555583Z" level=info msg="Fetched 985/985 events from the database" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:354"
time="2020-10-23T12:41:08.415731082Z" level=info msg="Fetched 992/992 events from the database" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:354"
time="2020-10-23T12:41:08.921652275Z" level=info msg="Fetched 996/996 events from the database" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:354"
time="2020-10-23T12:41:09.354788932Z" level=info msg="Fetched 983/983 events from the database" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:354"
time="2020-10-23T12:41:09.555737491Z" level=info msg="Fetched 983/983 events from the database" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:354"
time="2020-10-23T12:41:09.765426274Z" level=info msg="Fetched 983/983 events from the database" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:354"
time="2020-10-23T12:41:10.189953774Z" level=info msg="Fetched 984/984 events from the database" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:354"
time="2020-10-23T12:41:10.399783055Z" level=info msg="Fetched 985/985 events from the database" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:354"
time="2020-10-23T12:41:10.600950624Z" level=info msg="Fetched 985/985 events from the database" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:354"
time="2020-10-23T12:41:10.903210447Z" level=info msg="Fetched 992/992 events from the database" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:354"
time="2020-10-23T12:41:11.396803691Z" level=info msg="Fetched 996/996 events from the database" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:354"
time="2020-10-23T12:41:11.827816754Z" level=info msg="Fetched 983/983 events from the database" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:354"
time="2020-10-23T12:41:12.038797247Z" level=info msg="Fetched 983/983 events from the database" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:354"
time="2020-10-23T12:41:12.241799035Z" level=info msg="Fetched 983/983 events from the database" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:354"
time="2020-10-23T12:41:13.074612921Z" level=info msg="Fetched 984/984 events from the database" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:354"
time="2020-10-23T12:41:13.289194833Z" level=info msg="Fetched 985/985 events from the database" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:354"
time="2020-10-23T12:41:13.503880738Z" level=info msg="Fetched 985/985 events from the database" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:354"
time="2020-10-23T12:41:13.795752707Z" level=info msg="Fetched 992/992 events from the database" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:354"
time="2020-10-23T12:41:14.333981362Z" level=info msg="Fetched 996/996 events from the database" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:354"
time="2020-10-23T12:41:14.767834414Z" level=info msg="Fetched 983/983 events from the database" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:354"
time="2020-10-23T12:41:14.983447262Z" level=info msg="Fetched 983/983 events from the database" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:354"
time="2020-10-23T12:41:15.188106861Z" level=info msg="Fetched 983/983 events from the database" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:354"
time="2020-10-23T12:41:15.204432470Z" level=info msg="backfilled 95 events" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*Backfiller).backfillViaFederation" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:119" room_id="!LhdcUQKgKZCiDUQRGK:matrix.org"
time="2020-10-23T12:41:26.290635162Z" level=info msg="Fetching 1 missing state events (from 5 possible servers)" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*Backfiller).fetchAndStoreMissingEvents" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:183" context=missing
time="2020-10-23T12:41:28.293527729Z" level=info msg="returned 1 PDUs which made events [{Event:<nil> Error:Bad signature from \"conduit.rs\" with ID \"ed25519:key1\" SoftFail:false}]" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*Backfiller).fetchAndStoreMissingEvents" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:203" context=missing event_id="$S7k6DOYQyn9w_vGjYXZPnzvo8Hzr6Tyt3M9FFu29Gio" server=matrix.org
time="2020-10-23T12:41:28.293589506Z" level=warning msg="event failed PDU checks" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*Backfiller).fetchAndStoreMissingEvents" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:206" context=missing error="Bad signature from \"conduit.rs\" with ID \"ed25519:key1\"" event_id="$S7k6DOYQyn9w_vGjYXZPnzvo8Hzr6Tyt3M9FFu29Gio" server=matrix.org
time="2020-10-23T12:41:28.310329283Z" level=info msg="returned 1 PDUs which made events [{Event:<nil> Error:Bad signature from \"conduit.rs\" with ID \"ed25519:key1\" SoftFail:false}]" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*Backfiller).fetchAndStoreMissingEvents" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:203" context=missing event_id="$S7k6DOYQyn9w_vGjYXZPnzvo8Hzr6Tyt3M9FFu29Gio" server=kernel-error.com
time="2020-10-23T12:41:28.310367401Z" level=warning msg="event failed PDU checks" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*Backfiller).fetchAndStoreMissingEvents" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:206" context=missing error="Bad signature from \"conduit.rs\" with ID \"ed25519:key1\"" event_id="$S7k6DOYQyn9w_vGjYXZPnzvo8Hzr6Tyt3M9FFu29Gio" server=kernel-error.com
time="2020-10-23T12:41:28.382720823Z" level=info msg="returned 1 PDUs which made events [{Event:<nil> Error:Bad signature from \"conduit.rs\" with ID \"ed25519:key1\" SoftFail:false}]" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*Backfiller).fetchAndStoreMissingEvents" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:203" context=missing event_id="$S7k6DOYQyn9w_vGjYXZPnzvo8Hzr6Tyt3M9FFu29Gio" server=privacytools.io
time="2020-10-23T12:41:28.382765810Z" level=warning msg="event failed PDU checks" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*Backfiller).fetchAndStoreMissingEvents" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:206" context=missing error="Bad signature from \"conduit.rs\" with ID \"ed25519:key1\"" event_id="$S7k6DOYQyn9w_vGjYXZPnzvo8Hzr6Tyt3M9FFu29Gio" server=privacytools.io
time="2020-10-23T12:41:28.424831231Z" level=info msg="returned 1 PDUs which made events [{Event:<nil> Error:Bad signature from \"conduit.rs\" with ID \"ed25519:key1\" SoftFail:false}]" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*Backfiller).fetchAndStoreMissingEvents" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:203" context=missing event_id="$S7k6DOYQyn9w_vGjYXZPnzvo8Hzr6Tyt3M9FFu29Gio" server=lost-frequencies.eu
time="2020-10-23T12:41:28.424887864Z" level=warning msg="event failed PDU checks" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*Backfiller).fetchAndStoreMissingEvents" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:206" context=missing error="Bad signature from \"conduit.rs\" with ID \"ed25519:key1\"" event_id="$S7k6DOYQyn9w_vGjYXZPnzvo8Hzr6Tyt3M9FFu29Gio" server=lost-frequencies.eu
time="2020-10-23T12:41:28.590146053Z" level=info msg="returned 1 PDUs which made events [{Event:<nil> Error:Bad signature from \"conduit.rs\" with ID \"ed25519:key1\" SoftFail:false}]" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*Backfiller).fetchAndStoreMissingEvents" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:203" context=missing event_id="$S7k6DOYQyn9w_vGjYXZPnzvo8Hzr6Tyt3M9FFu29Gio" server=hispagatos.org
time="2020-10-23T12:41:28.590198689Z" level=warning msg="event failed PDU checks" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*Backfiller).fetchAndStoreMissingEvents" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:206" context=missing error="Bad signature from \"conduit.rs\" with ID \"ed25519:key1\"" event_id="$S7k6DOYQyn9w_vGjYXZPnzvo8Hzr6Tyt3M9FFu29Gio" server=hispagatos.org
time="2020-10-23T12:41:28.590219492Z" level=info msg="Persisting 0 new events" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*Backfiller).fetchAndStoreMissingEvents" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:220" context=missing
time="2020-10-23T12:41:28.596828004Z" level=error msg="backfillViaFederation: failed to get state entries for event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*Backfiller).backfillViaFederation" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:143" error="storage: state event IDs missing from the database (980 != 981)" event_id="$YMAYoxPOgtW6VzjCxJvNOUutCidtWiX9pWlSIxXOSDY"
time="2020-10-23T12:41:28.596883912Z" level=error msg="mreq.retrieveEvents failed" func=github.com/matrix-org/dendrite/syncapi/routing.OnIncomingMessagesRequest file="github.com/matrix-org/dendrite/syncapi/routing/messages.go:161" error="PerformBackfill failed: storage: state event IDs missing from the database (980 != 981)" req.id=SCK5zWiNXznF req.method=GET req.path="/_matrix/client/r0/rooms/!LhdcUQKgKZCiDUQRGK:matrix.org/messages" user_id="@pestdoktor:fentker.eu"
!YTvKGNlinIzlkMTVRl:matrix.org
time="2020-10-23T12:57:04.719455590Z" level=info msg="Fetched 0 events locally" func="github.com/matrix-org/dendrite/syncapi/routing.(*messagesReq).retrieveEvents" file="github.com/matrix-org/dendrite/syncapi/routing/messages.go:212" end="{{ []}}" req.id=2kBittwqud1j req.method=GET req.path="/_matrix/client/r0/rooms/!YTvKGNlinIzlkMTVRl:matrix.org/messages" start="{{ []}}" user_id="@pestdoktor:fentker.eu"
time="2020-10-23T12:57:04.730043124Z" level=info msg="ServersAtEvent history visibility not shared: world_readable" func=github.com/matrix-org/dendrite/roomserver/internal/perform.joinEventsFromHistoryVisibility file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:530"
time="2020-10-23T12:57:04.730074474Z" level=info msg="ServersAtEvent including 0 current events from history visibility" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).ServersAtEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:434"
time="2020-10-23T12:57:06.678023913Z" level=info msg="Fetching 1 key(s)" func="github.com/matrix-org/dendrite/signingkeyserver/internal.(*ServerKeyAPI).handleFetcherKeys" file="github.com/matrix-org/dendrite/signingkeyserver/internal/api.go:208" fetcher_name="perspective server matrix.org"
time="2020-10-23T12:57:20.607751624Z" level=info msg="Updated 1 of 17 key(s) in database (0 keys remaining)" func="github.com/matrix-org/dendrite/signingkeyserver/internal.(*ServerKeyAPI).handleFetcherKeys" file="github.com/matrix-org/dendrite/signingkeyserver/internal/api.go:262" fetcher_name="perspective server matrix.org"
time="2020-10-23T12:57:20.624046269Z" level=info msg="Fetching 1 key(s)" func="github.com/matrix-org/dendrite/signingkeyserver/internal.(*ServerKeyAPI).handleFetcherKeys" file="github.com/matrix-org/dendrite/signingkeyserver/internal/api.go:208" fetcher_name="perspective server matrix.org"
time="2020-10-23T12:57:20.649398956Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$7HsDdCLoYo4RdCrwrzcCVsxFl5XhW6WkswNU2m7Rwag"
time="2020-10-23T12:57:22.377591999Z" level=info msg="Updated 1 of 1 key(s) in database (0 keys remaining)" func="github.com/matrix-org/dendrite/signingkeyserver/internal.(*ServerKeyAPI).handleFetcherKeys" file="github.com/matrix-org/dendrite/signingkeyserver/internal/api.go:262" fetcher_name="perspective server matrix.org"
time="2020-10-23T12:57:24.074256102Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$CbpiGUo-UGlXopi8iDR6IWHRGmMK9EbCfx09JdVAZyU"
time="2020-10-23T12:57:26.730364813Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$9eAUys62CPqYTVboOkhWJazSIFiuU8QoCZPaMOhrz9U"
time="2020-10-23T12:57:28.234384153Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$E9T_XMUtWbXIHFwBmbieUV6BOV1xX1N8QAyRXBBRxi4"
time="2020-10-23T12:57:29.983731950Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$g2WrlVGhK0YDapUCoxug6G7adTHEYtDPetqcalf_uQc"
time="2020-10-23T12:57:31.716676675Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$Yl3w5duVD7MZQpIwhJOaLXkewt3fSrgk5cEqp5dGX2Q"
time="2020-10-23T12:57:33.407121467Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$L2_Ew77A_6lft-YwLvalH2zz6lsqpRY3C--VQrT9zb8"
time="2020-10-23T12:57:36.018868043Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$MFLb3J2UDPPwKCezK0LNFB3kkpbEyL-6L7LUFrKMbJE"
time="2020-10-23T12:57:37.550514844Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$w5wQ-DCoftCY9SDfDl20_LPY7e0_JT240xGjhLW-q54"
time="2020-10-23T12:57:39.065251339Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$hGQALip9dddpohFMt7pWotezfyFRmOlJyqLXUUxT620"
time="2020-10-23T12:57:41.407984394Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$6zWE1I6zpGehOXgHCgzHQ43ZWrH2dpmxYoZge3FUhog"
time="2020-10-23T12:57:43.061328768Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$LsL8LZ9AlbibCkEdBD8vuIA9J6QuHmiGJUjVVRjEz4o"
time="2020-10-23T12:57:44.561843252Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$JVSGnh3ldCdzauwRCW1NIHeuVZXTDpyiX9s6_0QuWRk"
time="2020-10-23T12:57:45.961929589Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$AuEy-oIMHdKxoFPjzsP8JG4Ih63LaBvu5Qgz1-PPrAc"
time="2020-10-23T12:57:47.774659981Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$1wW7bdhLjxHrtYMWCBylGsyaWUU07v48pyp6YsbvSgc"
time="2020-10-23T12:57:49.615050096Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$MeCjKH_BZRIiBHucnoGyU_NV1lyM7sNYiNKYnAChwGM"
time="2020-10-23T12:57:52.790753667Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$RzcpHg8wSo_TPQ0vUTO9TqUd5moI9Ou-TQXJmVwgH98"
time="2020-10-23T12:57:54.320716602Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$sCtTODAEITbqqA-ASiq00k1Mbb8nAN7MDaFjA6WuOvY"
time="2020-10-23T12:57:55.843533982Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$bx4ZWrfLMhh6YWGG4UFFF5uMntG0YnIXsoco4MxMdPE"
time="2020-10-23T12:57:57.233750558Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$P2P9r-MPCafQM8nnrWbmmcdT8YhkmdQAaf7xiP6Sahc"
time="2020-10-23T12:57:58.707818795Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$TWRvnXmVWOEMFeyjzbMyt9EWzeHgv4QdExp4z6nqWv8"
time="2020-10-23T12:58:00.438552919Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$vuRd-152IqHiAIjCgvUTGbpp8x03o8JbOHxCXcwiebs"
time="2020-10-23T12:58:02.668812157Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$mX1z6jtJKtscA3muIrkEogi4870kqEmb_70IbIjhJsA"
time="2020-10-23T12:58:04.140259200Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$kd1l2CGPofZ0owrwd9lyb0KWnRjqelmuqU92jmtgHQE"
time="2020-10-23T12:58:05.803854368Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$BVOv4D47nE6Awn3kP-7icbhUVX-ac1qIy1B6g2MYnEw"
time="2020-10-23T12:58:07.164738015Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$3aB0QWsipd3FjHvQ3Nj8LzDtAMqDl4gAAh5Tm_pFO14"
time="2020-10-23T12:58:11.590381903Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$y7FB2IfYucLoVMylUg0kYTp-fqqBuqzECjxk1qUmXgk"
time="2020-10-23T12:58:13.331917012Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$VOAGtuiec9Oqd9lbL5-AF_EZB4ZEPG-4V1rgEckWVrE"
time="2020-10-23T12:58:16.437661823Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$Je9hF3rVHmbYW4WqDWNJeHIVd0Tp-JUoQ4fnmfdkyl4"
time="2020-10-23T12:58:19.544803149Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$o1WQ8QYUzhyNlU87cWEUOVEkqfYFQZ7fBOWzn0KWnXI"
time="2020-10-23T12:58:21.121371084Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$xRTQ8vmKwSJ5KthcEmb0acjCpA7KicNwq3h8SrsO12M"
time="2020-10-23T12:58:22.573667767Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$3XQgQUOJUwiYeSZKtAcyrgElccpv8PjXdA2gsyMJTv0"
time="2020-10-23T12:58:25.304832269Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$8CtGk0ns2BmzFk-BbDWB_Rx2RFRv2GqGI6tQJhP6Fqc"
time="2020-10-23T12:58:26.802860233Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$4bDD1USiArHqfllYjMukhiOFFGavO_gaHWvfrGo1o64"
time="2020-10-23T12:58:36.029510077Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$hewAA3wcj2k1WWf_j6wbJKMCUBnoJFpKxwRvqR8Ln2c"
time="2020-10-23T12:58:40.270837086Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$RyB-RigfNfjOzHwKIFpLMbkzraBnH7wuzoQ2qBxRMOU"
time="2020-10-23T12:58:43.518654516Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$74diUEL5HP-o-SSt_wRmU2xu0BWtTfu6tfoO-9LCxyI"
time="2020-10-23T12:58:44.957794301Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$8vHDz2w8zn70xwuNeGzok1KZsAYxFbEuIf2Dq79H2hw"
time="2020-10-23T12:58:47.202027568Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$ftBAO51snq7g4MQv_8nyjjqfQd7KNZEl7KHXa57onLc"
time="2020-10-23T12:58:49.303979197Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$-qmpjiHT8SYCuc_CRkPj-lfuR4811DsWtY8GkqMtRsI"
time="2020-10-23T12:58:50.982268383Z" level=info msg="backfilled 100 events" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*Backfiller).backfillViaFederation" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:119" room_id="!YTvKGNlinIzlkMTVRl:matrix.org"
time="2020-10-23T12:58:51.413400350Z" level=info msg="Fetching 2 missing state events (from 5 possible servers)" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*Backfiller).fetchAndStoreMissingEvents" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:183" context=missing
time="2020-10-23T12:58:51.575179372Z" level=info msg="Fetching 1 key(s)" func="github.com/matrix-org/dendrite/signingkeyserver/internal.(*ServerKeyAPI).handleFetcherKeys" file="github.com/matrix-org/dendrite/signingkeyserver/internal/api.go:208" fetcher_name="perspective server matrix.org"
time="2020-10-23T12:58:53.584833574Z" level=info msg="Updated 1 of 2 key(s) in database (0 keys remaining)" func="github.com/matrix-org/dendrite/signingkeyserver/internal.(*ServerKeyAPI).handleFetcherKeys" file="github.com/matrix-org/dendrite/signingkeyserver/internal/api.go:262" fetcher_name="perspective server matrix.org"
time="2020-10-23T12:58:53.586683684Z" level=info msg="returned 1 PDUs which made events [{Event:<nil> Error:Bad signature from \"letsrulethe.world\" with ID \"ed25519:a_GsOc\" SoftFail:false}]" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*Backfiller).fetchAndStoreMissingEvents" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:203" context=missing event_id="$_4DJoDrJ5TfWrAQUQarjIjtva4Hj04lgureaQbGoHKg" server=matrix.org
time="2020-10-23T12:58:53.586732180Z" level=warning msg="event failed PDU checks" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*Backfiller).fetchAndStoreMissingEvents" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:206" context=missing error="Bad signature from \"letsrulethe.world\" with ID \"ed25519:a_GsOc\"" event_id="$_4DJoDrJ5TfWrAQUQarjIjtva4Hj04lgureaQbGoHKg" server=matrix.org
time="2020-10-23T12:58:53.777227294Z" level=info msg="Fetching 1 key(s)" func="github.com/matrix-org/dendrite/signingkeyserver/internal.(*ServerKeyAPI).handleFetcherKeys" file="github.com/matrix-org/dendrite/signingkeyserver/internal/api.go:208" fetcher_name="perspective server matrix.org"
time="2020-10-23T12:58:53.939372729Z" level=info msg="Updated 1 of 2 key(s) in database (0 keys remaining)" func="github.com/matrix-org/dendrite/signingkeyserver/internal.(*ServerKeyAPI).handleFetcherKeys" file="github.com/matrix-org/dendrite/signingkeyserver/internal/api.go:262" fetcher_name="perspective server matrix.org"
time="2020-10-23T12:58:53.941487990Z" level=info msg="returned 1 PDUs which made events [{Event:<nil> Error:Bad signature from \"michel-slm.name\" with ID \"ed25519:0\" SoftFail:false}]" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*Backfiller).fetchAndStoreMissingEvents" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:203" context=missing event_id="$Xg3Q7sHovdfPNuJ6MH2ABhEjHqqCKraokNzPdopnQk0" server=matrix.org
time="2020-10-23T12:58:53.941534653Z" level=warning msg="event failed PDU checks" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*Backfiller).fetchAndStoreMissingEvents" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:206" context=missing error="Bad signature from \"michel-slm.name\" with ID \"ed25519:0\"" event_id="$Xg3Q7sHovdfPNuJ6MH2ABhEjHqqCKraokNzPdopnQk0" server=matrix.org
time="2020-10-23T12:58:54.496992820Z" level=info msg="returned 1 PDUs which made events [{Event:<nil> Error:Bad signature from \"letsrulethe.world\" with ID \"ed25519:a_GsOc\" SoftFail:false}]" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*Backfiller).fetchAndStoreMissingEvents" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:203" context=missing event_id="$_4DJoDrJ5TfWrAQUQarjIjtva4Hj04lgureaQbGoHKg" server=dmnd.sh
time="2020-10-23T12:58:54.497066966Z" level=warning msg="event failed PDU checks" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*Backfiller).fetchAndStoreMissingEvents" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:206" context=missing error="Bad signature from \"letsrulethe.world\" with ID \"ed25519:a_GsOc\"" event_id="$_4DJoDrJ5TfWrAQUQarjIjtva4Hj04lgureaQbGoHKg" server=dmnd.sh
time="2020-10-23T12:58:54.559917411Z" level=info msg="returned 1 PDUs which made events [{Event:<nil> Error:Bad signature from \"michel-slm.name\" with ID \"ed25519:0\" SoftFail:false}]" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*Backfiller).fetchAndStoreMissingEvents" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:203" context=missing event_id="$Xg3Q7sHovdfPNuJ6MH2ABhEjHqqCKraokNzPdopnQk0" server=dmnd.sh
time="2020-10-23T12:58:54.559951871Z" level=warning msg="event failed PDU checks" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*Backfiller).fetchAndStoreMissingEvents" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:206" context=missing error="Bad signature from \"michel-slm.name\" with ID \"ed25519:0\"" event_id="$Xg3Q7sHovdfPNuJ6MH2ABhEjHqqCKraokNzPdopnQk0" server=dmnd.sh
time="2020-10-23T12:58:55.143841399Z" level=info msg="returned 1 PDUs which made events [{Event:<nil> Error:Bad signature from \"michel-slm.name\" with ID \"ed25519:0\" SoftFail:false}]" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*Backfiller).fetchAndStoreMissingEvents" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:203" context=missing event_id="$Xg3Q7sHovdfPNuJ6MH2ABhEjHqqCKraokNzPdopnQk0" server=raim.ist
time="2020-10-23T12:58:55.143913996Z" level=warning msg="event failed PDU checks" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*Backfiller).fetchAndStoreMissingEvents" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:206" context=missing error="Bad signature from \"michel-slm.name\" with ID \"ed25519:0\"" event_id="$Xg3Q7sHovdfPNuJ6MH2ABhEjHqqCKraokNzPdopnQk0" server=raim.ist
time="2020-10-23T12:58:55.322186658Z" level=info msg="returned 1 PDUs which made events [{Event:<nil> Error:Bad signature from \"letsrulethe.world\" with ID \"ed25519:a_GsOc\" SoftFail:false}]" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*Backfiller).fetchAndStoreMissingEvents" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:203" context=missing event_id="$_4DJoDrJ5TfWrAQUQarjIjtva4Hj04lgureaQbGoHKg" server=raim.ist
time="2020-10-23T12:58:55.322250715Z" level=warning msg="event failed PDU checks" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*Backfiller).fetchAndStoreMissingEvents" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:206" context=missing error="Bad signature from \"letsrulethe.world\" with ID \"ed25519:a_GsOc\"" event_id="$_4DJoDrJ5TfWrAQUQarjIjtva4Hj04lgureaQbGoHKg" server=raim.ist
time="2020-10-23T12:58:55.718343960Z" level=info msg="returned 1 PDUs which made events [{Event:<nil> Error:Bad signature from \"letsrulethe.world\" with ID \"ed25519:a_GsOc\" SoftFail:false}]" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*Backfiller).fetchAndStoreMissingEvents" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:203" context=missing event_id="$_4DJoDrJ5TfWrAQUQarjIjtva4Hj04lgureaQbGoHKg" server=matrix.m31.zone
time="2020-10-23T12:58:55.718405677Z" level=warning msg="event failed PDU checks" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*Backfiller).fetchAndStoreMissingEvents" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:206" context=missing error="Bad signature from \"letsrulethe.world\" with ID \"ed25519:a_GsOc\"" event_id="$_4DJoDrJ5TfWrAQUQarjIjtva4Hj04lgureaQbGoHKg" server=matrix.m31.zone
time="2020-10-23T12:58:55.975146212Z" level=info msg="returned 1 PDUs which made events [{Event:<nil> Error:Bad signature from \"michel-slm.name\" with ID \"ed25519:0\" SoftFail:false}]" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*Backfiller).fetchAndStoreMissingEvents" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:203" context=missing event_id="$Xg3Q7sHovdfPNuJ6MH2ABhEjHqqCKraokNzPdopnQk0" server=matrix.m31.zone
time="2020-10-23T12:58:55.975218988Z" level=warning msg="event failed PDU checks" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*Backfiller).fetchAndStoreMissingEvents" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:206" context=missing error="Bad signature from \"michel-slm.name\" with ID \"ed25519:0\"" event_id="$Xg3Q7sHovdfPNuJ6MH2ABhEjHqqCKraokNzPdopnQk0" server=matrix.m31.zone
time="2020-10-23T12:59:04.189983581Z" level=warning msg="failed to get event from server" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*Backfiller).fetchAndStoreMissingEvents" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:194" context=missing error="contents=[34 34] msg=Failed to GET JSON (hostname \"bsv.direct\" path \"/_matrix/federation/v1/event/$_4DJoDrJ5TfWrAQUQarjIjtva4Hj04lgureaQbGoHKg\"): \"\" code=404 wrapped= - (retry_after=0s, blacklisted=false)" event_id="$_4DJoDrJ5TfWrAQUQarjIjtva4Hj04lgureaQbGoHKg" server=bsv.direct
time="2020-10-23T12:59:04.428957238Z" level=info msg="returned 1 PDUs which made events [{Event:<nil> Error:Bad signature from \"michel-slm.name\" with ID \"ed25519:0\" SoftFail:false}]" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*Backfiller).fetchAndStoreMissingEvents" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:203" context=missing event_id="$Xg3Q7sHovdfPNuJ6MH2ABhEjHqqCKraokNzPdopnQk0" server=bsv.direct
time="2020-10-23T12:59:04.429003274Z" level=warning msg="event failed PDU checks" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*Backfiller).fetchAndStoreMissingEvents" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:206" context=missing error="Bad signature from \"michel-slm.name\" with ID \"ed25519:0\"" event_id="$Xg3Q7sHovdfPNuJ6MH2ABhEjHqqCKraokNzPdopnQk0" server=bsv.direct
time="2020-10-23T12:59:04.429066100Z" level=info msg="Persisting 0 new events" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*Backfiller).fetchAndStoreMissingEvents" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:220" context=missing
time="2020-10-23T12:59:04.496070322Z" level=error msg="backfillViaFederation: failed to get state entries for event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*Backfiller).backfillViaFederation" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:143" error="storage: state event IDs missing from the database (7547 != 7549)" event_id="$L2_Ew77A_6lft-YwLvalH2zz6lsqpRY3C--VQrT9zb8"
time="2020-10-23T12:59:04.496132589Z" level=error msg="mreq.retrieveEvents failed" func=github.com/matrix-org/dendrite/syncapi/routing.OnIncomingMessagesRequest file="github.com/matrix-org/dendrite/syncapi/routing/messages.go:161" error="PerformBackfill failed: storage: state event IDs missing from the database (7547 != 7549)" req.id=2kBittwqud1j req.method=GET req.path="/_matrix/client/r0/rooms/!YTvKGNlinIzlkMTVRl:matrix.org/messages" user_id="@pestdoktor:fentker.eu"
!yomrOFwgFXzmeMAbzX:matrix.org
time="2020-10-23T14:20:06.177312534Z" level=info msg="Fetched 50 events locally" func="github.com/matrix-org/dendrite/syncapi/routing.(*messagesReq).retrieveEvents" file="github.com/matrix-org/dendrite/syncapi/routing/messages.go:212" end="{{ []}}" req.id=ExUvJWv55dRy req.method=GET req.path="/_matrix/client/r0/rooms/!yomrOFwgFXzmeMAbzX:matrix.org/messages" start="{{ []}}" user_id="@pestdoktor:fentker.eu"
time="2020-10-23T14:20:06.180251969Z" level=info msg="ServersAtEvent history visibility not shared: world_readable" func=github.com/matrix-org/dendrite/roomserver/internal/perform.joinEventsFromHistoryVisibility file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:530"
time="2020-10-23T14:20:06.180300683Z" level=info msg="ServersAtEvent including 0 current events from history visibility" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).ServersAtEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:434"
time="2020-10-23T14:20:06.954253713Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$z4chuLrzB2VTIyG2PXdbp6k_xuHXOLjs937DfvyWooI"
time="2020-10-23T14:20:07.297156803Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$-d6GpijXJqX_HTUrWacTNYvbb-ybjd5RtNYUH5-Aj_o"
time="2020-10-23T14:20:08.212559831Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$qdEEIjVtaU45_1hfgLe5ElFWyXhwKQkhk4X68a9EI98"
time="2020-10-23T14:20:08.802346275Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$HP-trZEH93jsIrkq8LFqvyJbdz8aPgL2A3-VdPYjI7k"
time="2020-10-23T14:20:09.181216433Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$IGUbYG-UxMMfd1PyofHbax8M6YtQ6BSVz7lA2FH1x3E"
time="2020-10-23T14:20:09.596041579Z" level=info msg="Requesting /state_ids at event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*backfillRequester).StateIDsBeforeEvent" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:290" event_id="$I04fyKdlNV0cJ8FvGENYYmFF5KK8h25SoDE3k0NrlHY"
time="2020-10-23T14:20:09.959187524Z" level=info msg="backfilled 100 events" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*Backfiller).backfillViaFederation" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:119" room_id="!yomrOFwgFXzmeMAbzX:matrix.org"
time="2020-10-23T14:20:10.206274722Z" level=info msg="Fetching 1 missing state events (from 5 possible servers)" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*Backfiller).fetchAndStoreMissingEvents" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:183" context=missing
time="2020-10-23T14:20:10.769322879Z" level=info msg="Fetching 1 key(s)" func="github.com/matrix-org/dendrite/signingkeyserver/internal.(*ServerKeyAPI).handleFetcherKeys" file="github.com/matrix-org/dendrite/signingkeyserver/internal/api.go:208" fetcher_name="perspective server matrix.org"
time="2020-10-23T14:20:11.321975279Z" level=info msg="Updated 1 of 2 key(s) in database (0 keys remaining)" func="github.com/matrix-org/dendrite/signingkeyserver/internal.(*ServerKeyAPI).handleFetcherKeys" file="github.com/matrix-org/dendrite/signingkeyserver/internal/api.go:262" fetcher_name="perspective server matrix.org"
time="2020-10-23T14:20:11.323349166Z" level=info msg="returned 1 PDUs which made events [{Event:<nil> Error:gomatrixserverlib: key with ID \"ed25519:a_pqvl\" for \"matrix.riedel.dev\" not valid at 1576114959176 SoftFail:false}]" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*Backfiller).fetchAndStoreMissingEvents" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:203" context=missing event_id="$PJ4qVwBpYGo5z0-FVeXQpFtN9-FdeALuoz8W4lqk-co" server=matrix.org
time="2020-10-23T14:20:11.323390088Z" level=warning msg="event failed PDU checks" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*Backfiller).fetchAndStoreMissingEvents" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:206" context=missing error="gomatrixserverlib: key with ID \"ed25519:a_pqvl\" for \"matrix.riedel.dev\" not valid at 1576114959176" event_id="$PJ4qVwBpYGo5z0-FVeXQpFtN9-FdeALuoz8W4lqk-co" server=matrix.org
time="2020-10-23T14:20:12.089731710Z" level=info msg="Fetching 1 key(s)" func="github.com/matrix-org/dendrite/signingkeyserver/internal.(*ServerKeyAPI).handleFetcherKeys" file="github.com/matrix-org/dendrite/signingkeyserver/internal/api.go:208" fetcher_name="perspective server matrix.org"
time="2020-10-23T14:20:12.371052385Z" level=info msg="Updated 1 of 2 key(s) in database (0 keys remaining)" func="github.com/matrix-org/dendrite/signingkeyserver/internal.(*ServerKeyAPI).handleFetcherKeys" file="github.com/matrix-org/dendrite/signingkeyserver/internal/api.go:262" fetcher_name="perspective server matrix.org"
time="2020-10-23T14:20:12.374012071Z" level=info msg="returned 1 PDUs which made events [{Event:<nil> Error:gomatrixserverlib: key with ID \"ed25519:a_pqvl\" for \"matrix.riedel.dev\" not valid at 1576114959176 SoftFail:false}]" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*Backfiller).fetchAndStoreMissingEvents" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:203" context=missing event_id="$PJ4qVwBpYGo5z0-FVeXQpFtN9-FdeALuoz8W4lqk-co" server=togart.de
time="2020-10-23T14:20:12.374186260Z" level=warning msg="event failed PDU checks" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*Backfiller).fetchAndStoreMissingEvents" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:206" context=missing error="gomatrixserverlib: key with ID \"ed25519:a_pqvl\" for \"matrix.riedel.dev\" not valid at 1576114959176" event_id="$PJ4qVwBpYGo5z0-FVeXQpFtN9-FdeALuoz8W4lqk-co" server=togart.de
time="2020-10-23T14:20:12.742886835Z" level=info msg="Fetching 1 key(s)" func="github.com/matrix-org/dendrite/signingkeyserver/internal.(*ServerKeyAPI).handleFetcherKeys" file="github.com/matrix-org/dendrite/signingkeyserver/internal/api.go:208" fetcher_name="perspective server matrix.org"
time="2020-10-23T14:20:13.057531255Z" level=info msg="Updated 1 of 2 key(s) in database (0 keys remaining)" func="github.com/matrix-org/dendrite/signingkeyserver/internal.(*ServerKeyAPI).handleFetcherKeys" file="github.com/matrix-org/dendrite/signingkeyserver/internal/api.go:262" fetcher_name="perspective server matrix.org"
time="2020-10-23T14:20:13.059313878Z" level=info msg="returned 1 PDUs which made events [{Event:<nil> Error:gomatrixserverlib: key with ID \"ed25519:a_pqvl\" for \"matrix.riedel.dev\" not valid at 1576114959176 SoftFail:false}]" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*Backfiller).fetchAndStoreMissingEvents" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:203" context=missing event_id="$PJ4qVwBpYGo5z0-FVeXQpFtN9-FdeALuoz8W4lqk-co" server=snopyta.org
time="2020-10-23T14:20:13.059398813Z" level=warning msg="event failed PDU checks" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*Backfiller).fetchAndStoreMissingEvents" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:206" context=missing error="gomatrixserverlib: key with ID \"ed25519:a_pqvl\" for \"matrix.riedel.dev\" not valid at 1576114959176" event_id="$PJ4qVwBpYGo5z0-FVeXQpFtN9-FdeALuoz8W4lqk-co" server=snopyta.org
time="2020-10-23T14:20:13.059467021Z" level=warning msg="failed to get event from server" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*Backfiller).fetchAndStoreMissingEvents" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:194" context=missing error=" - (retry_after=0s, blacklisted=true)" event_id="$PJ4qVwBpYGo5z0-FVeXQpFtN9-FdeALuoz8W4lqk-co" server=dendrite.shipp.dev
time="2020-10-23T14:20:13.258257121Z" level=info msg="Fetching 1 key(s)" func="github.com/matrix-org/dendrite/signingkeyserver/internal.(*ServerKeyAPI).handleFetcherKeys" file="github.com/matrix-org/dendrite/signingkeyserver/internal/api.go:208" fetcher_name="perspective server matrix.org"
time="2020-10-23T14:20:13.610101271Z" level=info msg="Updated 1 of 2 key(s) in database (0 keys remaining)" func="github.com/matrix-org/dendrite/signingkeyserver/internal.(*ServerKeyAPI).handleFetcherKeys" file="github.com/matrix-org/dendrite/signingkeyserver/internal/api.go:262" fetcher_name="perspective server matrix.org"
time="2020-10-23T14:20:13.613181786Z" level=info msg="returned 1 PDUs which made events [{Event:<nil> Error:gomatrixserverlib: key with ID \"ed25519:a_pqvl\" for \"matrix.riedel.dev\" not valid at 1576114959176 SoftFail:false}]" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*Backfiller).fetchAndStoreMissingEvents" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:203" context=missing event_id="$PJ4qVwBpYGo5z0-FVeXQpFtN9-FdeALuoz8W4lqk-co" server=matrix.nomagic.uk
time="2020-10-23T14:20:13.613344896Z" level=warning msg="event failed PDU checks" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*Backfiller).fetchAndStoreMissingEvents" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:206" context=missing error="gomatrixserverlib: key with ID \"ed25519:a_pqvl\" for \"matrix.riedel.dev\" not valid at 1576114959176" event_id="$PJ4qVwBpYGo5z0-FVeXQpFtN9-FdeALuoz8W4lqk-co" server=matrix.nomagic.uk
time="2020-10-23T14:20:13.613455328Z" level=info msg="Persisting 0 new events" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*Backfiller).fetchAndStoreMissingEvents" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:220" context=missing
time="2020-10-23T14:20:13.639517877Z" level=error msg="backfillViaFederation: failed to get state entries for event" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*Backfiller).backfillViaFederation" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_backfill.go:143" error="storage: state event IDs missing from the database (755 != 756)" event_id="$jWrYkJ_I7w3Jz41pFbmb3vGswO9ZvUPapjlynWhAVz0"
time="2020-10-23T14:20:13.639690316Z" level=error msg="mreq.retrieveEvents failed" func=github.com/matrix-org/dendrite/syncapi/routing.OnIncomingMessagesRequest file="github.com/matrix-org/dendrite/syncapi/routing/messages.go:161" error="PerformBackfill failed: storage: state event IDs missing from the database (755 != 756)" req.id=ExUvJWv55dRy req.method=GET req.path="/_matrix/client/r0/rooms/!yomrOFwgFXzmeMAbzX:matrix.org/messages" user_id="@pestdoktor:fentker.eu"
While trying to capture a complete log from a fresh deployment for !LhdcUQKgKZCiDUQRGK:matrix.org, I got different behaviour (end token not advancing):
while (!=s $token t0_0) { token = (curl -G -H 'Authorization: Bearer '$access_token -d from=$token -d dir=b -d limit=100 "https://d3.fentker.eu:8448/_matrix/client/r0/rooms/"(print $room | /usr/lib/plan9/bin/urlencode)"/messages" | jq .end | from-json); echo $token}
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 73989 0 73989 0 0 468 0 --:--:-- 0:02:37 --:--:-- 16478
t12776_1003
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 75007 0 75007 0 0 551 0 --:--:-- 0:02:16 --:--:-- 15698
t12682_1102
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 75007 0 75007 0 0 488 0 --:--:-- 0:02:33 --:--:-- 18424
t12682_1102
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 75007 0 75007 0 0 417 0 --:--:-- 0:02:59 --:--:-- 21906
t12682_1102
Complete log from fresh deployment for !OGEhHVWSdvArJzumhm:matrix.org: Monolith.log
Dupe of https://github.com/matrix-org/dendrite/issues/1544 - it's the failure to get the state event that causes this to fail, not the failing sigs on conduit.rs e.g storage: state event IDs missing from the database (7547 != 7549)
Background information
Steps to reproduce
Logs