Background information

Dendrite version or git SHA: 0.6.2
Monolith or Polylith?: monolith
SQLite3 or Postgres?: postgres
Running in Docker?: no
go version: 1.16.10
Client used (if applicable): element-web

Description

What is the problem: fails to join #go-lang:matrix.org and #dendrite:matrix.org
Who is affected: single user on my dendrite test server.
How is this bug manifesting: join event is propagated to remote servers (user shows up), but join is never completed
When did this first appear: -

Steps to reproduce

new install of dendrite 0.6.2 with empty db
attempt to join #go-lang:matrix.org and #dendrite:matrix.org
spinning forever in element-web, new join attempt says join is in progress (while events are still processed) or starts new failing attempt.

time="2022-02-06T13:51:36.874685465Z" level=info msg="User requested to room join" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*Joiner).PerformJoin" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_join.go:59" room_id="#dendrite:matrix.org" servers="[]" user_id="@rbn:dendrite.collec.tf"
time="2022-02-06T13:53:10.280276162Z" level=info msg="Checking event signatures for 7444 events of room state" func="github.com/matrix-org/gomatrixserverlib.(*RespState).Check" file="github.com/matrix-org/gomatrixserverlib@v0.0.0-20220204110702-c559d2019275/federationtypes.go:515" context=missing
time="2022-02-06T13:54:30.808965085Z" level=warning msg="Failed to request keys from fetcher" func=github.com/matrix-org/gomatrixserverlib.KeyRing.VerifyJSONs file="github.com/matrix-org/gomatrixserverlib@v0.0.0-20220204110702-c559d2019275/keyring.go:263" context=missing error="gomatrixserverlib: unable to lookup server keys: Post \"matrix://matrix.org/_matrix/key/v2/query\": dial tcp: i/o timeout" fetcher="perspective server matrix.org"
time="2022-02-06T13:54:36.464257570Z" level=error msg="Failed to fetch key for server" func="github.com/matrix-org/gomatrixserverlib.(*DirectKeyFetcher).FetchKeys.func1" file="github.com/matrix-org/gomatrixserverlib@v0.0.0-20220204110702-c559d2019275/keyring.go:497" context=missing error="Post \"matrix://dendrite.rom.dog/_matrix/key/v2/query\": dial tcp 129.153.209.212:8448: connect: no route to host" fetcher=DirectKeyFetcher
time="2022-02-06T13:54:36.464361168Z" level=warning msg="Failed to retrieve any keys" func=github.com/matrix-org/gomatrixserverlib.KeyRing.VerifyJSONs file="github.com/matrix-org/gomatrixserverlib@v0.0.0-20220204110702-c559d2019275/keyring.go:268" context=missing fetcher=DirectKeyFetcher
time="2022-02-06T13:57:24.144458659Z" level=warning msg="Failed to request keys from fetcher" func=github.com/matrix-org/gomatrixserverlib.KeyRing.VerifyJSONs file="github.com/matrix-org/gomatrixserverlib@v0.0.0-20220204110702-c559d2019275/keyring.go:263" context=missing error="gomatrixserverlib: unable to lookup server keys: Post \"matrix://matrix.org/_matrix/key/v2/query\": dial tcp: i/o timeout" fetcher="perspective server matrix.org"
time="2022-02-06T13:58:03.495808786Z" level=warning msg="Failed to retrieve any keys" func=github.com/matrix-org/gomatrixserverlib.KeyRing.VerifyJSONs file="github.com/matrix-org/gomatrixserverlib@v0.0.0-20220204110702-c559d2019275/keyring.go:268" context=missing fetcher="perspective server matrix.org"
time="2022-02-06T13:58:24.120391769Z" level=error msg="Failed to fetch key for server" func="github.com/matrix-org/gomatrixserverlib.(*DirectKeyFetcher).FetchKeys.func1" file="github.com/matrix-org/gomatrixserverlib@v0.0.0-20220204110702-c559d2019275/keyring.go:497" context=missing error="Post \"matrix://testuser-dendrite.duckdns.org/_matrix/key/v2/query\": dial tcp 78.46.90.98:8448: connect: connection refused" fetcher=DirectKeyFetcher
time="2022-02-06T13:58:24.120532855Z" level=warning msg="Failed to retrieve any keys" func=github.com/matrix-org/gomatrixserverlib.KeyRing.VerifyJSONs file="github.com/matrix-org/gomatrixserverlib@v0.0.0-20220204110702-c559d2019275/keyring.go:268" context=missing fetcher=DirectKeyFetcher
time="2022-02-06T13:58:37.171866527Z" level=warning msg="Signature validation failed for event \"$nUsQL4zbQ1c4w6PBo_YZnY4PrIRAflvmMtjjuSZ6w0A\"" func="github.com/matrix-org/gomatrixserverlib.(*RespState).Check" file="github.com/matrix-org/gomatrixserverlib@v0.0.0-20220204110702-c559d2019275/federationtypes.go:525" error="Bad signature from \"proprietary.lol\" with ID \"ed25519:bCfvh8\""
time="2022-02-06T13:58:37.171933868Z" level=warning msg="Signature validation failed for event \"$vQXj4wh5ikIq0d3o4YP-R9JSfxVLyYeN1wN4lRbnEME\"" func="github.com/matrix-org/gomatrixserverlib.(*RespState).Check" file="github.com/matrix-org/gomatrixserverlib@v0.0.0-20220204110702-c559d2019275/federationtypes.go:525" error="gomatrixserverlib: key with ID \"ed25519:8KV0b9\" for \"thejoyo.com\" not valid at 1625171310648"
time="2022-02-06T13:58:37.171965512Z" level=warning msg="Signature validation failed for event \"$tA36TpKlYOBPgd4iwgSxwdzDMzTQEN_9gndo_jsiGDc\"" func="github.com/matrix-org/gomatrixserverlib.(*RespState).Check" file="github.com/matrix-org/gomatrixserverlib@v0.0.0-20220204110702-c559d2019275/federationtypes.go:525" error="gomatrixserverlib: key with ID \"ed25519:8KV0b9\" for \"thejoyo.com\" not valid at 1624480683292"
time="2022-02-06T13:58:37.171994155Z" level=warning msg="Signature validation failed for event \"$opnx4MbHiyjpVXLLAUGqeS9pTesLku75vhyZv63aESU\"" func="github.com/matrix-org/gomatrixserverlib.(*RespState).Check" file="github.com/matrix-org/gomatrixserverlib@v0.0.0-20220204110702-c559d2019275/federationtypes.go:525" error="gomatrixserverlib: key with ID \"ed25519:a_iCck\" for \"050821.xyz\" not valid at 1639749673013"
time="2022-02-06T13:58:37.172021135Z" level=warning msg="Signature validation failed for event \"$LZUBb6InCd-LAX8XK2VSxhRAsKE6v_M8bipjP6sMEAQ\"" func="github.com/matrix-org/gomatrixserverlib.(*RespState).Check" file="github.com/matrix-org/gomatrixserverlib@v0.0.0-20220204110702-c559d2019275/federationtypes.go:525" error="Bad signature from \"manuel.life\" with ID \"ed25519:0\""
time="2022-02-06T13:58:37.172046914Z" level=warning msg="Signature validation failed for event \"$Oqt_bOfPQ3jCPz-hN0DshyPESb5AV6qxtX2NuIMaNZE\"" func="github.com/matrix-org/gomatrixserverlib.(*RespState).Check" file="github.com/matrix-org/gomatrixserverlib@v0.0.0-20220204110702-c559d2019275/federationtypes.go:525" error="gomatrixserverlib: key with ID \"ed25519:8KV0b9\" for \"thejoyo.com\" not valid at 1625169545873"
time="2022-02-06T13:58:37.172073315Z" level=warning msg="Signature validation failed for event \"$xj97EFD4GLIu7KZXYxzyncGuAaVCk028c6hqS075ZjU\"" func="github.com/matrix-org/gomatrixserverlib.(*RespState).Check" file="github.com/matrix-org/gomatrixserverlib@v0.0.0-20220204110702-c559d2019275/federationtypes.go:525" error="gomatrixserverlib: key with ID \"ed25519:a_rRAI\" for \"satterfield.us\" not valid at 1611292731225"
time="2022-02-06T13:58:37.172100148Z" level=warning msg="Signature validation failed for event \"$In4o1J4STij69Ku25heRi7qp5qYUYIYDomUUKgRglvU\"" func="github.com/matrix-org/gomatrixserverlib.(*RespState).Check" file="github.com/matrix-org/gomatrixserverlib@v0.0.0-20220204110702-c559d2019275/federationtypes.go:525" error="gomatrixserverlib: could not download key for \"dendrite.rom.dog\""
time="2022-02-06T13:58:37.172145098Z" level=warning msg="Signature validation failed for event \"$sdBf2fEEiJ2IqtlrMehO_2-d3tZduksGZuWrtILDUL4\"" func="github.com/matrix-org/gomatrixserverlib.(*RespState).Check" file="github.com/matrix-org/gomatrixserverlib@v0.0.0-20220204110702-c559d2019275/federationtypes.go:525" error="gomatrixserverlib: key with ID \"ed25519:a_WRob\" for \"hispagatos.org\" not valid at 1596997206194"
time="2022-02-06T13:58:37.172210785Z" level=warning msg="Signature validation failed for event \"$H2TVPESLexnOXFxL5u0kd9VxA01_J-lhtNwxcAQWCtM\"" func="github.com/matrix-org/gomatrixserverlib.(*RespState).Check" file="github.com/matrix-org/gomatrixserverlib@v0.0.0-20220204110702-c559d2019275/federationtypes.go:525" error="Bad signature from \"proprietary.lol\" with ID \"ed25519:bCfvh8\""
time="2022-02-06T13:58:37.172276011Z" level=warning msg="Signature validation failed for event \"$afFI9Z9DlXdrM8B4n3tGrL0KCvaaP8iw0iNBcwxSiBM\"" func="github.com/matrix-org/gomatrixserverlib.(*RespState).Check" file="github.com/matrix-org/gomatrixserverlib@v0.0.0-20220204110702-c559d2019275/federationtypes.go:525" error="gomatrixserverlib: key with ID \"ed25519:NWbDrM\" for \"cripslock.undef.tools\" not valid at 1616293484998"
time="2022-02-06T13:58:37.172338901Z" level=warning msg="Signature validation failed for event \"$yf0_5aVue2aEPNxRiqpANEwv1NHNuPv2mCK3wBwpUVI\"" func="github.com/matrix-org/gomatrixserverlib.(*RespState).Check" file="github.com/matrix-org/gomatrixserverlib@v0.0.0-20220204110702-c559d2019275/federationtypes.go:525" error="gomatrixserverlib: key with ID \"ed25519:8KV0b9\" for \"thejoyo.com\" not valid at 1624480111032"
time="2022-02-06T13:58:37.172397334Z" level=warning msg="Signature validation failed for event \"$Y2YWwBkfsU4HI2L6fQxCdi3GxOCoyEGiVHjhmm8arOs\"" func="github.com/matrix-org/gomatrixserverlib.(*RespState).Check" file="github.com/matrix-org/gomatrixserverlib@v0.0.0-20220204110702-c559d2019275/federationtypes.go:525" error="gomatrixserverlib: key with ID \"ed25519:NWbDrM\" for \"cripslock.undef.tools\" not valid at 1616281905787"
time="2022-02-06T13:58:37.172460346Z" level=warning msg="Signature validation failed for event \"$zmaN5OQvckXrjh_VcQdUtJt6bUt5JJTKaudyU8_GAT4\"" func="github.com/matrix-org/gomatrixserverlib.(*RespState).Check" file="github.com/matrix-org/gomatrixserverlib@v0.0.0-20220204110702-c559d2019275/federationtypes.go:525" error="gomatrixserverlib: key with ID \"ed25519:NWbDrM\" for \"cripslock.undef.tools\" not valid at 1616293465447"
time="2022-02-06T13:58:37.172544108Z" level=warning msg="Signature validation failed for event \"$PJ4qVwBpYGo5z0-FVeXQpFtN9-FdeALuoz8W4lqk-co\"" func="github.com/matrix-org/gomatrixserverlib.(*RespState).Check" file="github.com/matrix-org/gomatrixserverlib@v0.0.0-20220204110702-c559d2019275/federationtypes.go:525" error="gomatrixserverlib: key with ID \"ed25519:a_pqvl\" for \"matrix.riedel.dev\" not valid at 1576114959176"
time="2022-02-06T13:58:37.172613311Z" level=warning msg="Signature validation failed for event \"$dpKzDoIlxEiSeatagYxiaNVTD8iw5z1yLl_8MB3S0vU\"" func="github.com/matrix-org/gomatrixserverlib.(*RespState).Check" file="github.com/matrix-org/gomatrixserverlib@v0.0.0-20220204110702-c559d2019275/federationtypes.go:525" error="Bad signature from \"bcow.xyz\" with ID \"ed25519:2bpeHI\""
time="2022-02-06T13:58:37.172661426Z" level=warning msg="Signature validation failed for event \"$Qw1xH1YDUn_HW4UW72wKz2ifdQpTL1pGkwf-OH9VLUE\"" func="github.com/matrix-org/gomatrixserverlib.(*RespState).Check" file="github.com/matrix-org/gomatrixserverlib@v0.0.0-20220204110702-c559d2019275/federationtypes.go:525" error="Bad signature from \"bcow.xyz\" with ID \"ed25519:2bpeHI\""
time="2022-02-06T13:58:37.172689578Z" level=warning msg="Signature validation failed for event \"$vLzEYeff1nwRFpSfzdU06bDV-7I5gQJO5XehfDrwiA8\"" func="github.com/matrix-org/gomatrixserverlib.(*RespState).Check" file="github.com/matrix-org/gomatrixserverlib@v0.0.0-20220204110702-c559d2019275/federationtypes.go:525" error="Bad signature from \"yiff.social\" with ID \"ed25519:L1RXQh\""
time="2022-02-06T13:58:37.172745412Z" level=warning msg="Signature validation failed for event \"$5NVCP7Qpel04r7pQq_FGL-g3aWozYbvEt2DTxh1wqqw\"" func="github.com/matrix-org/gomatrixserverlib.(*RespState).Check" file="github.com/matrix-org/gomatrixserverlib@v0.0.0-20220204110702-c559d2019275/federationtypes.go:525" error="Bad signature from \"bcow.xyz\" with ID \"ed25519:2bpeHI\""
time="2022-02-06T13:58:37.172790891Z" level=warning msg="Signature validation failed for event \"$ujuHayQfxcEF4KDOcX1skapZAX5crBq5c_oUrY1nLzs\"" func="github.com/matrix-org/gomatrixserverlib.(*RespState).Check" file="github.com/matrix-org/gomatrixserverlib@v0.0.0-20220204110702-c559d2019275/federationtypes.go:525" error="gomatrixserverlib: key with ID \"ed25519:a_ddyx\" for \"1bit.se\" not valid at 1607793481969"
time="2022-02-06T13:58:37.172835247Z" level=warning msg="Signature validation failed for event \"$L_Pxtj7qwKG-nBIA_ScRDqNh6k8Wgf1R0HWY_EtzAGk\"" func="github.com/matrix-org/gomatrixserverlib.(*RespState).Check" file="github.com/matrix-org/gomatrixserverlib@v0.0.0-20220204110702-c559d2019275/federationtypes.go:525" error="gomatrixserverlib: key with ID \"ed25519:0\" for \"nwca.xyz\" not valid at 1592406093989"
time="2022-02-06T13:58:37.172879463Z" level=warning msg="Signature validation failed for event \"$fk9Zum02DzZHetLONYUqJ25o3nAJ9S81bmsKd-H7Dko\"" func="github.com/matrix-org/gomatrixserverlib.(*RespState).Check" file="github.com/matrix-org/gomatrixserverlib@v0.0.0-20220204110702-c559d2019275/federationtypes.go:525" error="gomatrixserverlib: key with ID \"ed25519:8KV0b9\" for \"thejoyo.com\" not valid at 1624479888688"
time="2022-02-06T13:58:37.172923299Z" level=warning msg="Signature validation failed for event \"$hhSQL0cWn7_fweP05_Im6dJvjydbecQ5ZrzN5rhZx2c\"" func="github.com/matrix-org/gomatrixserverlib.(*RespState).Check" file="github.com/matrix-org/gomatrixserverlib@v0.0.0-20220204110702-c559d2019275/federationtypes.go:525" error="gomatrixserverlib: key with ID \"ed25519:8KV0b9\" for \"thejoyo.com\" not valid at 1625169326522"
time="2022-02-06T13:58:37.172974583Z" level=warning msg="Signature validation failed for event \"$coc3xMJT9gSPpP0HZ_sMVLCDWnyEr06ORaQrkg9n-aA\"" func="github.com/matrix-org/gomatrixserverlib.(*RespState).Check" file="github.com/matrix-org/gomatrixserverlib@v0.0.0-20220204110702-c559d2019275/federationtypes.go:525" error="Bad signature from \"yiff.social\" with ID \"ed25519:L1RXQh\""
time="2022-02-06T13:58:37.173004263Z" level=warning msg="Signature validation failed for event \"$DU-1BZvFyxcTYdLdshN5Iiwrl0Bion_ymYPG5kzAvCU\"" func="github.com/matrix-org/gomatrixserverlib.(*RespState).Check" file="github.com/matrix-org/gomatrixserverlib@v0.0.0-20220204110702-c559d2019275/federationtypes.go:525" error="Bad signature from \"proprietary.lol\" with ID \"ed25519:bCfvh8\""
time="2022-02-06T13:58:37.173042613Z" level=warning msg="Signature validation failed for event \"$0ES4-kHuqyWenIQN_NFe02mH31i1ohp9n7yQcBG1YKc\"" func="github.com/matrix-org/gomatrixserverlib.(*RespState).Check" file="github.com/matrix-org/gomatrixserverlib@v0.0.0-20220204110702-c559d2019275/federationtypes.go:525" error="Bad signature from \"manuel.life\" with ID \"ed25519:0\""
time="2022-02-06T13:58:37.173079307Z" level=warning msg="Signature validation failed for event \"$UELQFbozuqvT1W1d612ubs6YxHfvhLMxjTocnbDeo-I\"" func="github.com/matrix-org/gomatrixserverlib.(*RespState).Check" file="github.com/matrix-org/gomatrixserverlib@v0.0.0-20220204110702-c559d2019275/federationtypes.go:525" error="gomatrixserverlib: key with ID \"ed25519:a_rRAI\" for \"satterfield.us\" not valid at 1611291871075"
time="2022-02-06T13:58:37.173116199Z" level=warning msg="Signature validation failed for event \"$4zICntTOmCIuHjCgcj_6yK2WpyneES6G-419cbb9Ngs\"" func="github.com/matrix-org/gomatrixserverlib.(*RespState).Check" file="github.com/matrix-org/gomatrixserverlib@v0.0.0-20220204110702-c559d2019275/federationtypes.go:525" error="gomatrixserverlib: key with ID \"ed25519:a_WRob\" for \"hispagatos.org\" not valid at 1576626668422"
time="2022-02-06T13:58:37.173153928Z" level=warning msg="Signature validation failed for event \"$yNaAliPolhDitOYu0arjawZbIe8Av-g1T1CMc1EJ_1o\"" func="github.com/matrix-org/gomatrixserverlib.(*RespState).Check" file="github.com/matrix-org/gomatrixserverlib@v0.0.0-20220204110702-c559d2019275/federationtypes.go:525" error="gomatrixserverlib: key with ID \"ed25519:8KV0b9\" for \"thejoyo.com\" not valid at 1625170070703"
time="2022-02-06T13:58:37.173206594Z" level=warning msg="Signature validation failed for event \"$cvjQu_m2cE1C7TtlSpEbpAJpx9c8avlwlq5SF_GPvOY\"" func="github.com/matrix-org/gomatrixserverlib.(*RespState).Check" file="github.com/matrix-org/gomatrixserverlib@v0.0.0-20220204110702-c559d2019275/federationtypes.go:525" error="gomatrixserverlib: key with ID \"ed25519:8KV0b9\" for \"thejoyo.com\" not valid at 1624480844485"
time="2022-02-06T13:58:37.173252463Z" level=warning msg="Signature validation failed for event \"$TWODd9VPdZOMn5zia2RfMwTVyeMNm5_KfpPTVAJ5zos\"" func="github.com/matrix-org/gomatrixserverlib.(*RespState).Check" file="github.com/matrix-org/gomatrixserverlib@v0.0.0-20220204110702-c559d2019275/federationtypes.go:525" error="gomatrixserverlib: key with ID \"ed25519:1orNdp\" for \"otsuka.haus\" not valid at 1636129968358"
time="2022-02-06T13:58:37.173299120Z" level=warning msg="Signature validation failed for event \"$Av-ZvCLQXtPDviPO6O_7LKFGERF3c_twlXwHxrX81t0\"" func="github.com/matrix-org/gomatrixserverlib.(*RespState).Check" file="github.com/matrix-org/gomatrixserverlib@v0.0.0-20220204110702-c559d2019275/federationtypes.go:525" error="gomatrixserverlib: key with ID \"ed25519:0\" for \"nwca.xyz\" not valid at 1592407546706"
time="2022-02-06T13:58:37.173356801Z" level=warning msg="Signature validation failed for event \"$UAe3OpyJWe3h6YC-wFRJOP6LF10Yj3mmbDAqvBosk3Y\"" func="github.com/matrix-org/gomatrixserverlib.(*RespState).Check" file="github.com/matrix-org/gomatrixserverlib@v0.0.0-20220204110702-c559d2019275/federationtypes.go:525" error="gomatrixserverlib: key with ID \"ed25519:a_iCck\" for \"050821.xyz\" not valid at 1639820924257"
time="2022-02-06T13:58:37.173403740Z" level=warning msg="Signature validation failed for event \"$fdt5MO-aYFTVG17zAwWBCglWUYnUagFYXfrgB_obxCw\"" func="github.com/matrix-org/gomatrixserverlib.(*RespState).Check" file="github.com/matrix-org/gomatrixserverlib@v0.0.0-20220204110702-c559d2019275/federationtypes.go:525" error="Bad signature from \"bcow.xyz\" with ID \"ed25519:2bpeHI\""
time="2022-02-06T13:58:37.173438840Z" level=warning msg="Signature validation failed for event \"$ZEDC7EHXMgwzIbrieNi0w-S15VBfAoikf7M3uCrb7Jc\"" func="github.com/matrix-org/gomatrixserverlib.(*RespState).Check" file="github.com/matrix-org/gomatrixserverlib@v0.0.0-20220204110702-c559d2019275/federationtypes.go:525" error="gomatrixserverlib: key with ID \"ed25519:a_WRob\" for \"hispagatos.org\" not valid at 1578677304783"
time="2022-02-06T13:58:37.173482676Z" level=warning msg="Signature validation failed for event \"$zisn4mEm2_IqZ-3wIuUTsn7qpTN31yeI2do8NtSHt_I\"" func="github.com/matrix-org/gomatrixserverlib.(*RespState).Check" file="github.com/matrix-org/gomatrixserverlib@v0.0.0-20220204110702-c559d2019275/federationtypes.go:525" error="Bad signature from \"proprietary.lol\" with ID \"ed25519:bCfvh8\""
time="2022-02-06T13:58:37.173529233Z" level=warning msg="Signature validation failed for event \"$7xFvGFT8AsNilDhpfLuV7_qIIHM5Rv84j4XwwjPSud8\"" func="github.com/matrix-org/gomatrixserverlib.(*RespState).Check" file="github.com/matrix-org/gomatrixserverlib@v0.0.0-20220204110702-c559d2019275/federationtypes.go:525" error="Bad signature from \"manuel.life\" with ID \"ed25519:0\""
time="2022-02-06T13:58:37.173572330Z" level=warning msg="Signature validation failed for event \"$Zr_n3zOge7q3DfBYpjFKC9pUdWSSRpVIXz-FY_12LO4\"" func="github.com/matrix-org/gomatrixserverlib.(*RespState).Check" file="github.com/matrix-org/gomatrixserverlib@v0.0.0-20220204110702-c559d2019275/federationtypes.go:525" error="gomatrixserverlib: key with ID \"ed25519:a_QChh\" for \"quincystrange.com\" not valid at 1605559323478"
time="2022-02-06T13:58:37.173615262Z" level=warning msg="Signature validation failed for event \"$-3HbQ3RkqyG0QmN21EpE8rENhyuGV3w04IaQJZ_HA7U\"" func="github.com/matrix-org/gomatrixserverlib.(*RespState).Check" file="github.com/matrix-org/gomatrixserverlib@v0.0.0-20220204110702-c559d2019275/federationtypes.go:525" error="gomatrixserverlib: key with ID \"ed25519:a_ddyx\" for \"1bit.se\" not valid at 1610379720212"
time="2022-02-06T13:58:37.173680078Z" level=warning msg="Signature validation failed for event \"$TQ_uGyD8qRJmFl6sBv0OhNlV6o7qPz2nESAhpzPpJe4\"" func="github.com/matrix-org/gomatrixserverlib.(*RespState).Check" file="github.com/matrix-org/gomatrixserverlib@v0.0.0-20220204110702-c559d2019275/federationtypes.go:525" error="gomatrixserverlib: could not download key for \"testuser-dendrite.duckdns.org\""
time="2022-02-06T13:58:37.173723516Z" level=warning msg="Signature validation failed for event \"$b70p5CMIUZRkutxmy-mX2YjgG34QwwzZDpFg9xqW6fE\"" func="github.com/matrix-org/gomatrixserverlib.(*RespState).Check" file="github.com/matrix-org/gomatrixserverlib@v0.0.0-20220204110702-c559d2019275/federationtypes.go:525" error="Bad signature from \"toke.dk\" with ID \"ed25519:xPlL9W\""
time="2022-02-06T13:58:37.173759698Z" level=warning msg="Signature validation failed for event \"$1LCTj6oPYWbxCGBnCLf4w9IEh27CitNPOHiYWCOoLNk\"" func="github.com/matrix-org/gomatrixserverlib.(*RespState).Check" file="github.com/matrix-org/gomatrixserverlib@v0.0.0-20220204110702-c559d2019275/federationtypes.go:525" error="gomatrixserverlib: key with ID \"ed25519:a_QChh\" for \"quincystrange.com\" not valid at 1607032247837"
time="2022-02-06T13:59:02.329573951Z" level=warning msg="Discarding 42 auth/state event(s) due to invalid signatures" func="github.com/matrix-org/gomatrixserverlib.(*RespState).Check" file="github.com/matrix-org/gomatrixserverlib@v0.0.0-20220204110702-c559d2019275/federationtypes.go:549" context=missing
time="2022-02-06T13:59:13.809501594Z" level=warning msg="Auth event $s0aNapxPDZFPpV8jjKb6WylujsQX2feITeU2H6HldS8 rejected" func="github.com/matrix-org/dendrite/roomserver/internal/input.(*Inputer).fetchAuthEvents" file="github.com/matrix-org/dendrite/roomserver/internal/input/input_events.go:454" context=missing error="eventauth: sender \"@abuse:matrix.org\" is not allowed to send event. 50 < 100" event_id="$n5OLkHun_GvxmsNuwLEGrT7W3atCH4ElZMQ86P8N2Pw" kind=KindOutlier origin=matrix.org room_id="!yomrOFwgFXzmeMAbzX:matrix.org" type=m.room.member
time="2022-02-06T13:59:13.933285380Z" level=warning msg="Roomserver failed to process sync event" func="github.com/matrix-org/dendrite/roomserver/internal/input.(*Inputer).InputRoomEvents.func1" file="github.com/matrix-org/dendrite/roomserver/internal/input/input.go:239" error="missing auth event $coc3xMJT9gSPpP0HZ_sMVLCDWnyEr06ORaQrkg9n-aA" event_id="$n5OLkHun_GvxmsNuwLEGrT7W3atCH4ElZMQ86P8N2Pw" room_id="!yomrOFwgFXzmeMAbzX:matrix.org"
time="2022-02-06T13:59:13.933418379Z" level=warning msg="Failed to join room through server" func="github.com/matrix-org/dendrite/federationapi/internal.(*FederationInternalAPI).PerformJoin" file="github.com/matrix-org/dendrite/federationapi/internal/perform.go:101" error="roomserverAPI.SendEventWithState: InputRoomEventsResponse: missing auth event $coc3xMJT9gSPpP0HZ_sMVLCDWnyEr06ORaQrkg9n-aA" room_id="!yomrOFwgFXzmeMAbzX:matrix.org" server_name=matrix.org
time="2022-02-06T13:59:13.933487544Z" level=error msg="failed to join user \"@rbn:dendrite.collec.tf\" to room \"!yomrOFwgFXzmeMAbzX:matrix.org\" through 1 server(s): last error roomserverAPI.SendEventWithState: InputRoomEventsResponse: missing auth event $coc3xMJT9gSPpP0HZ_sMVLCDWnyEr06ORaQrkg9n-aA" func="github.com/matrix-org/dendrite/federationapi/internal.(*FederationInternalAPI).PerformJoin" file="github.com/matrix-org/dendrite/federationapi/internal/perform.go:129"
time="2022-02-06T13:59:13.933544706Z" level=error msg="Failed to join room" func="github.com/matrix-org/dendrite/roomserver/internal/perform.(*Joiner).PerformJoin" file="github.com/matrix-org/dendrite/roomserver/internal/perform/perform_join.go:62" error="5 : roomserverAPI.SendEventWithState: InputRoomEventsResponse: missing auth event $coc3xMJT9gSPpP0HZ_sMVLCDWnyEr06ORaQrkg9n-aA" room_id="#dendrite:matrix.org" servers="[]" user_id="@rbn:dendrite.collec.tf"

I think I am experiencing the same problem. I cannot join any (even small) federated rooms and Element says "Joining" forever. I tried joining the german pine64 room because it only had about 12 people, but that still didn't work. I am running dendrite monolith v0.9.1 with postgres outside of docker

Some logs:

```log Aug 16 17:39:09 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:39:09.154662390Z" level=warning msg="Failed to retrieve any keys" context=missing fetcher=DirectKeyFetcher Aug 16 17:39:09 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:39:09.154362590Z" level=error msg="Failed to fetch key for server" context=missing error="Post \"matrix://snd.ddnss.de/_matrix/key/v2/query\": dial tcp 89.244.221.185:8448: connect: no route to host" fetcher=DirectKeyFetcher Aug 16 17:39:07 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:39:07.413395770Z" level=warning msg="Failed to retrieve any keys" context=missing fetcher=DirectKeyFetcher Aug 16 17:39:07 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:39:07.413265663Z" level=error msg="Failed to fetch key for server" context=missing error="Post \"matrix://die-falsche-wahrheit.de/_matrix/key/v2/query\": dial tcp: lookup die-falsche-wahrheit.de: Temporary failure in name resolution" fetcher=DirectKeyFetcher Aug 16 17:39:06 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:39:06.291626287Z" level=warning msg="Failed to retrieve any keys" context=missing fetcher=DirectKeyFetcher Aug 16 17:39:06 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:39:06.291430986Z" level=error msg="Failed to fetch key for server" context=missing error="Post \"matrix://chat.anarchyserver.crabdance.com/_matrix/key/v2/query\": dial tcp: lookup chat.anarchyserver.crabdance.com: Temporary failure in name resolution" fetcher=DirectKeyFetcher Aug 16 17:38:57 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:38:57.329616474Z" level=warning msg="Failed to retrieve any keys" context=missing fetcher=DirectKeyFetcher Aug 16 17:38:57 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:38:57.329503116Z" level=error msg="Failed to fetch key for server" context=missing error="contents=[60 104 116 109 108 62 10 60 104 101 97 100 62 60 116 105 116 108 101 62 53 48 50 32 66 97 100 32 71 97 116 101 119 97 121 60 47 116 105 116 108 101 62 60 47 104 101 97 100 62 10 60 98 111 100 121 62 10 60 99 101 110 116 101 114 62 60 104 49 62 53 48 50 32 66 97 100 32 71 97 116 101 119 97 121 60 47 104 49 62 60 47 99 101 110 116 101 114 62 10 60 104 114 62 60 99 101 110 116 101 114 62 110 103 105 110 120 47 49 46 49 56 46 48 32 40 85 98 117 110 116 117 41 60 47 99 101 110 116 101 114 62 10 60 47 98 111 100 121 62 10 60 47 104 116 109 108 62 10] msg=Failed to POST JSON (hostname \"matrix.tanaybiradar.com\" path \"/_matrix/key/v2/query\"): \n502 Bad Gateway\n\n

502 Bad Gateway

nginx/1.18.0 (Ubuntu)\n\n\n code=502 wrapped=" fetcher=DirectKeyFetcher Aug 16 17:38:49 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:38:49.335690206Z" level=warning msg="Failed to retrieve any keys" context=missing fetcher=DirectKeyFetcher Aug 16 17:38:49 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:38:49.335540318Z" level=error msg="Failed to fetch key for server" context=missing error="Post \"matrix://necenglab.com/_matrix/key/v2/query\": dial tcp 20.58.160.126:8448: i/o timeout" fetcher=DirectKeyFetcher Aug 16 17:38:25 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:38:25.819248157Z" level=warning msg="Failed to retrieve any keys" context=missing fetcher=DirectKeyFetcher Aug 16 17:38:25 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:38:25.818893544Z" level=error msg="Failed to fetch key for server" context=missing error="Post \"matrix://atomize.me/_matrix/key/v2/query\": dial tcp 98.202.48.169:8448: i/o timeout" fetcher=DirectKeyFetcher Aug 16 17:38:14 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:38:14.437185055Z" level=warning msg="Failed to retrieve any keys" context=missing fetcher=DirectKeyFetcher Aug 16 17:38:14 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:38:14.437100199Z" level=error msg="Failed to fetch key for server" context=missing error="Post \"matrix://matrix.guja.nl/_matrix/key/v2/query\": x509: certificate has expired or is not yet valid: current time 2022-08-16T17:38:14-04:00 is after 2022-08-01T18:07:05Z" fetcher=DirectKeyFetcher Aug 16 17:38:11 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:38:11.008189965Z" level=warning msg="Failed to retrieve any keys" context=missing fetcher=DirectKeyFetcher Aug 16 17:38:11 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:38:11.008089582Z" level=error msg="Failed to fetch key for server" context=missing error="Post \"matrix://sammydepresso.ems.host/_matrix/key/v2/query\": dial tcp: lookup sammydepresso.ems.host: Temporary failure in name resolution" fetcher=DirectKeyFetcher Aug 16 17:37:55 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:37:55.747784833Z" level=warning msg="Failed to retrieve any keys" context=missing fetcher=DirectKeyFetcher Aug 16 17:37:55 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:37:55.747706987Z" level=error msg="Failed to fetch key for server" context=missing error="Post \"matrix://matrix.anunnaki-lab.com/_matrix/key/v2/query\": dial tcp: lookup matrix.anunnaki-lab.com: Temporary failure in name resolution" fetcher=DirectKeyFetcher Aug 16 17:37:45 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:37:45.037968434Z" level=warning msg="Failed to retrieve any keys" context=missing fetcher=DirectKeyFetcher Aug 16 17:37:45 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:37:45.037864056Z" level=error msg="Failed to fetch key for server" context=missing error="Post \"matrix://matrix.kubelt.com/_matrix/key/v2/query\": context deadline exceeded" fetcher=DirectKeyFetcher Aug 16 17:37:14 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:37:14.891842969Z" level=warning msg="Failed to retrieve any keys" context=missing fetcher=DirectKeyFetcher Aug 16 17:37:14 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:37:14.891748624Z" level=error msg="Failed to fetch key for server" context=missing error="Post \"matrix://matrix.streamcoh.com/_matrix/key/v2/query\": dial tcp 70.112.118.213:8448: i/o timeout" fetcher=DirectKeyFetcher Aug 16 17:36:54 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:36:54.446763043Z" level=warning msg="Failed to retrieve any keys" context=missing fetcher=DirectKeyFetcher Aug 16 17:36:54 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:36:54.446650079Z" level=error msg="Failed to fetch key for server" context=missing error="Post \"matrix://chaos.institute/_matrix/key/v2/query\": dial tcp: lookup chaos.institute: no such host" fetcher=DirectKeyFetcher Aug 16 17:36:48 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:36:48.342624556Z" level=warning msg="Failed to retrieve any keys" context=missing fetcher=DirectKeyFetcher Aug 16 17:36:48 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:36:48.342445914Z" level=error msg="Failed to fetch key for server" context=missing error="Post \"matrix://chat.darkfire.sh/_matrix/key/v2/query\": dial tcp: lookup chat.darkfire.sh: Temporary failure in name resolution" fetcher=DirectKeyFetcher Aug 16 17:36:43 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:36:43.768785556Z" level=warning msg="Failed to retrieve any keys" context=missing fetcher=DirectKeyFetcher Aug 16 17:36:43 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:36:43.768658537Z" level=error msg="Failed to fetch key for server" context=missing error="Post \"matrix://matrix.chrisbohn.org/_matrix/key/v2/query\": dial tcp 69.137.120.234:8448: i/o timeout" fetcher=DirectKeyFetcher Aug 16 17:36:22 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:36:22.356110120Z" level=warning msg="Failed to retrieve any keys" context=missing fetcher=DirectKeyFetcher Aug 16 17:36:22 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:36:22.355714887Z" level=error msg="Failed to fetch key for server" context=missing error="Post \"matrix://matrixtest.chadwaltercummings.me/_matrix/key/v2/query\": dial tcp: lookup matrixtest.chadwaltercummings.me: i/o timeout" fetcher=DirectKeyFetcher Aug 16 17:36:00 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:36:00.263696616Z" level=warning msg="Failed to retrieve any keys" context=missing fetcher=DirectKeyFetcher Aug 16 17:36:00 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:36:00.263575022Z" level=error msg="Failed to fetch key for server" context=missing error="Post \"matrix://matrix.kubelt.com/_matrix/key/v2/query\": context deadline exceeded" fetcher=DirectKeyFetcher Aug 16 17:35:30 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:35:30.225156006Z" level=warning msg="Failed to retrieve any keys" context=missing fetcher=DirectKeyFetcher Aug 16 17:35:30 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:35:30.225001572Z" level=error msg="Failed to fetch key for server" context=missing error="Post \"matrix://fedi.io/_matrix/key/v2/query\": dial tcp 3.64.163.50:8448: i/o timeout" fetcher=DirectKeyFetcher Aug 16 17:35:10 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:35:10.278692340Z" level=info msg=DeviceListUpdater.Update context=missing deleted=false device_id=IAKYXNITVW display_name="Element Desktop (Linux)" prev_ids="[4971727550]" prev_ids_exist=false stream_id=4971731004 user_id="@btc-satanist:matrix.org" Aug 16 17:35:08 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:35:08.964796076Z" level=info msg=DeviceListUpdater.Update context=missing deleted=false device_id=IAKYXNITVW display_name="Element Desktop (Linux)" prev_ids="[]" prev_ids_exist=false stream_id=4971727550 user_id="@btc-satanist:matrix.org" Aug 16 17:35:08 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:35:08.753855969Z" level=info msg=DeviceListUpdater.Update context=missing deleted=false device_id=XRKNIIKKBQ display_name="" prev_ids="[4971714340]" prev_ids_exist=false stream_id=4971728909 user_id="@jwh4j4ez25q:matrix.org" Aug 16 17:35:05 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:35:05.305282264Z" level=warning msg="Failed to retrieve any keys" context=missing fetcher=DirectKeyFetcher Aug 16 17:35:05 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:35:05.305146254Z" level=error msg="Failed to fetch key for server" context=missing error="Post \"matrix://matrix.omerfurkandemircioglu.com.tr/_matrix/key/v2/query\": context deadline exceeded" fetcher=DirectKeyFetcher Aug 16 17:34:34 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:34:34.674349560Z" level=warning msg="Failed to retrieve any keys" context=missing fetcher=DirectKeyFetcher Aug 16 17:34:34 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:34:34.674228562Z" level=error msg="Failed to fetch key for server" context=missing error="Post \"matrix://going-on.tech/_matrix/key/v2/query\": dial tcp 174.138.18.61:8448: i/o timeout" fetcher=DirectKeyFetcher Aug 16 17:34:13 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:34:13.003010190Z" level=warning msg="Failed to retrieve any keys" context=missing fetcher=DirectKeyFetcher Aug 16 17:34:13 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:34:13.002892857Z" level=error msg="Failed to fetch key for server" context=missing error="Post \"matrix://derhartekern.space/_matrix/key/v2/query\": dial tcp 84.137.210.99:8448: connect: no route to host" fetcher=DirectKeyFetcher Aug 16 17:34:11 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:34:11.155946143Z" level=warning msg="Failed to retrieve any keys" context=missing fetcher=DirectKeyFetcher Aug 16 17:34:11 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:34:11.155773037Z" level=error msg="Failed to fetch key for server" context=missing error="Post \"matrix://kroxelikrax.se/_matrix/key/v2/query\": context deadline exceeded" fetcher=DirectKeyFetcher Aug 16 17:33:40 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:33:40.025979233Z" level=warning msg="Failed to retrieve any keys" context=missing fetcher=DirectKeyFetcher Aug 16 17:33:40 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:33:40.025896378Z" level=error msg="Failed to fetch key for server" context=missing error="Post \"matrix://m.welterde.de/_matrix/key/v2/query\": dial tcp [2001:678:c70:c08:12:20fa::]:8448: connect: network is unreachable" fetcher=DirectKeyFetcher Aug 16 17:33:38 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:33:38.821080472Z" level=warning msg="Failed to retrieve any keys" context=missing fetcher=DirectKeyFetcher Aug 16 17:33:38 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:33:38.820893724Z" level=error msg="Failed to fetch key for server" context=missing error="Post \"matrix://greluc.de/_matrix/key/v2/query\": context deadline exceeded" fetcher=DirectKeyFetcher Aug 16 17:33:08 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:33:08.553690131Z" level=warning msg="Failed to retrieve any keys" context=missing fetcher=DirectKeyFetcher Aug 16 17:33:08 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:33:08.553552153Z" level=error msg="Failed to fetch key for server" context=missing error="Post \"matrix://nerdtreff.it/_matrix/key/v2/query\": dial tcp 185.216.179.248:8448: i/o timeout" fetcher=DirectKeyFetcher Aug 16 17:32:47 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:32:47.163867560Z" level=warning msg="Failed to retrieve any keys" context=missing fetcher=DirectKeyFetcher Aug 16 17:32:47 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:32:47.163363550Z" level=error msg="Failed to fetch key for server" context=missing error="Post \"matrix://rubberducker.xyz/_matrix/key/v2/query\": dial tcp: lookup rubberducker.xyz: Temporary failure in name resolution" fetcher=DirectKeyFetcher Aug 16 17:32:45 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:32:45.677981507Z" level=warning msg="Failed to retrieve any keys" context=missing fetcher=DirectKeyFetcher Aug 16 17:32:45 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:32:45.677881525Z" level=error msg="Failed to fetch key for server" context=missing error="Post \"matrix://matrixtest.chadwaltercummings.me/_matrix/key/v2/query\": dial tcp: lookup matrixtest.chadwaltercummings.me: Temporary failure in name resolution" fetcher=DirectKeyFetcher Aug 16 17:32:40 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:32:40.378711197Z" level=warning msg="Failed to retrieve any keys" context=missing fetcher=DirectKeyFetcher Aug 16 17:32:40 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:32:40.378581220Z" level=error msg="Failed to fetch key for server" context=missing error="Post \"matrix://tmchat.de/_matrix/key/v2/query\": context deadline exceeded" fetcher=DirectKeyFetcher Aug 16 17:32:10 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:32:10.278834119Z" level=warning msg="Failed to retrieve any keys" context=missing fetcher=DirectKeyFetcher Aug 16 17:32:10 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:32:10.278750053Z" level=error msg="Failed to fetch key for server" context=missing error="Post \"matrix://m.clmv.de/_matrix/key/v2/query\": dial tcp: lookup m.clmv.de: No address associated with hostname" fetcher=DirectKeyFetcher Aug 16 17:32:08 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:32:08.794614068Z" level=warning msg="Failed to retrieve any keys" context=missing fetcher=DirectKeyFetcher Aug 16 17:32:08 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:32:08.794399941Z" level=error msg="Failed to fetch key for server" context=missing error="Post \"matrix://deepcover.io/_matrix/key/v2/query\": dial tcp: lookup deepcover.io: i/o timeout" fetcher=DirectKeyFetcher Aug 16 17:31:47 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:31:47.882309401Z" level=warning msg="Failed to retrieve any keys" context=missing fetcher=DirectKeyFetcher Aug 16 17:31:47 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:31:47.882117852Z" level=error msg="Failed to fetch key for server" context=missing error="Post \"matrix://familie-vogdt.de/_matrix/key/v2/query\": context deadline exceeded" fetcher=DirectKeyFetcher Aug 16 17:31:17 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:31:17.799310388Z" level=warning msg="Failed to retrieve any keys" context=missing fetcher=DirectKeyFetcher Aug 16 17:31:17 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:31:17.798965537Z" level=error msg="Failed to fetch key for server" context=missing error="Post \"matrix://alnoimy.club/_matrix/key/v2/query\": dial tcp 82.165.101.82:8448: i/o timeout" fetcher=DirectKeyFetcher Aug 16 17:30:56 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:30:56.934691286Z" level=warning msg="Failed to retrieve any keys" context=missing fetcher=DirectKeyFetcher Aug 16 17:30:56 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:30:56.934592877Z" level=error msg="Failed to fetch key for server" context=missing error="Post \"matrix://hug.us.to/_matrix/key/v2/query\": dial tcp: lookup hug.us.to: Temporary failure in name resolution" fetcher=DirectKeyFetcher Aug 16 17:30:46 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:30:46.659020270Z" level=warning msg="Failed to retrieve any keys" context=missing fetcher=DirectKeyFetcher Aug 16 17:30:46 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:30:46.658924881Z" level=error msg="Failed to fetch key for server" context=missing error="Post \"matrix://fl0r1s.net/_matrix/key/v2/query\": dial tcp: lookup fl0r1s.net: Temporary failure in name resolution" fetcher=DirectKeyFetcher Aug 16 17:30:45 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:30:45.573920353Z" level=warning msg="Failed to retrieve any keys" context=missing fetcher=DirectKeyFetcher Aug 16 17:30:45 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:30:45.573727054Z" level=error msg="Failed to fetch key for server" context=missing error="Post \"matrix://matrixtest.chadwaltercummings.me/_matrix/key/v2/query\": dial tcp: lookup matrixtest.chadwaltercummings.me: Temporary failure in name resolution" fetcher=DirectKeyFetcher Aug 16 17:30:32 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:30:32.612389801Z" level=warning msg="Failed to retrieve any keys" context=missing fetcher=DirectKeyFetcher Aug 16 17:30:32 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:30:32.612262725Z" level=error msg="Failed to fetch key for server" context=missing error="Post \"matrix://matrix.sunberryonline.com/_matrix/key/v2/query\": dial tcp 140.82.40.144:8448: i/o timeout" fetcher=DirectKeyFetcher Aug 16 17:30:18 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:30:18.083172305Z" level=info msg=DeviceListUpdater.Update context=missing deleted=false device_id=EVPKLDHZVP display_name="" prev_ids="[4971686209]" prev_ids_exist=false stream_id=4971714340 user_id="@jwh4j4ez25q:matrix.org" Aug 16 17:30:12 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:30:12.157863562Z" level=warning msg="Failed to retrieve any keys" context=missing fetcher=DirectKeyFetcher Aug 16 17:30:12 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:30:12.157755373Z" level=error msg="Failed to fetch key for server" context=missing error="Post \"matrix://matrix.jackpoint.virtadpt.net/_matrix/key/v2/query\": dial tcp 104.236.109.172:8448: i/o timeout" fetcher=DirectKeyFetcher Aug 16 17:30:02 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:30:02.315336271Z" level=info msg=DeviceListUpdater.Update context=missing deleted=false device_id=UXCSEAVKOF display_name="app.element.io (Edge, macOS)" prev_ids="[4942915972]" prev_ids_exist=false stream_id=4971713359 user_id="@fgfhmnrtry4r4t:matrix.org" Aug 16 17:29:51 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:29:51.678555678Z" level=warning msg="Failed to retrieve any keys" context=missing fetcher=DirectKeyFetcher Aug 16 17:29:51 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:29:51.678349839Z" level=error msg="Failed to fetch key for server" context=missing error="Post \"matrix://thegaymatrix.org/_matrix/key/v2/query\": context deadline exceeded" fetcher=DirectKeyFetcher Aug 16 17:29:39 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:29:39.719868542Z" level=info msg=DeviceListUpdater.Update context=missing deleted=true device_id=WXUTPFHSNC display_name="" prev_ids="[4971701636]" prev_ids_exist=false stream_id=4971710916 user_id="@istori:matrix.org" Aug 16 17:29:29 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:29:29.101063060Z" level=warning msg="Failed to retrieve any keys" context=missing fetcher=DirectKeyFetcher Aug 16 17:29:29 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:29:29.100875881Z" level=error msg="Failed to fetch key for server" context=missing error="Post \"matrix://janpodrouzek.cz/_matrix/key/v2/query\": context deadline exceeded" fetcher=DirectKeyFetcher Aug 16 17:29:05 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:29:05.291312862Z" level=info msg=DeviceListUpdater.Update context=missing deleted=false device_id=MKMWPNZVVP display_name=Mobile prev_ids="[]" prev_ids_exist=false stream_id=4971705285 user_id="@carylynechan:matrix.org" Aug 16 17:28:54 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:28:54.380538055Z" level=warning msg="Failed to retrieve any keys" context=missing fetcher=DirectKeyFetcher Aug 16 17:28:54 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:28:54.380431284Z" level=error msg="Failed to fetch key for server" context=missing error="Post \"matrix://matrix.baarck.eu/_matrix/key/v2/query\": dial tcp [2a00:6020:49a3:ea00:211:32ff:fee5:8488]:8448: connect: network is unreachable" fetcher=DirectKeyFetcher Aug 16 17:28:53 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:28:53.834345807Z" level=warning msg="Failed to retrieve any keys" context=missing fetcher=DirectKeyFetcher Aug 16 17:28:53 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:28:53.833893767Z" level=error msg="Failed to fetch key for server" context=missing error="Post \"matrix://brachmann.me/_matrix/key/v2/query\": context deadline exceeded" fetcher=DirectKeyFetcher Aug 16 17:28:23 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:28:23.817330668Z" level=warning msg="Failed to retrieve any keys" context=missing fetcher=DirectKeyFetcher Aug 16 17:28:23 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:28:23.817109469Z" level=error msg="Failed to fetch key for server" context=missing error="Post \"matrix://rubberducker.xyz/_matrix/key/v2/query\": dial tcp: lookup rubberducker.xyz: Temporary failure in name resolution" fetcher=DirectKeyFetcher Aug 16 17:28:10 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:28:10.120199869Z" level=warning msg="Failed to retrieve any keys" context=missing fetcher=DirectKeyFetcher Aug 16 17:28:10 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:28:10.120109417Z" level=error msg="Failed to fetch key for server" context=missing error="Post \"matrix://matrix.aidanwebster.uk/_matrix/key/v2/query\": context deadline exceeded" fetcher=DirectKeyFetcher Aug 16 17:27:38 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:27:38.273484922Z" level=warning msg="Failed to retrieve any keys" context=missing fetcher=DirectKeyFetcher Aug 16 17:27:38 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:27:38.273407559Z" level=error msg="Failed to fetch key for server" context=missing error="contents=[52 48 52 32 112 97 103 101 32 110 111 116 32 102 111 117 110 100 10] msg=Failed to POST JSON (hostname \"mgoddon.de\" path \"/_matrix/key/v2/query\"): 404 page not found\n code=404 wrapped=" fetcher=DirectKeyFetcher Aug 16 17:27:37 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:27:37.654164490Z" level=warning msg="Failed to retrieve any keys" context=missing fetcher=DirectKeyFetcher Aug 16 17:27:37 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:27:37.653860524Z" level=error msg="Failed to fetch key for server" context=missing error="Post \"matrix://dc7342.im/_matrix/key/v2/query\": dial tcp 46.146.231.187:8448: i/o timeout" fetcher=DirectKeyFetcher Aug 16 17:27:37 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:27:37.094125730Z" level=info msg=DeviceListUpdater.Update context=missing deleted=false device_id=WXUTPFHSNC display_name="FluffyChat android" prev_ids="[4971684407]" prev_ids_exist=false stream_id=4971701636 user_id="@istori:matrix.org" Aug 16 17:27:16 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:27:16.000033061Z" level=warning msg="Failed to retrieve any keys" context=missing fetcher=DirectKeyFetcher Aug 16 17:27:15 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:27:15.999821552Z" level=error msg="Failed to fetch key for server" context=missing error="Post \"matrix://invisiblemoon.xyz/_matrix/key/v2/query\": dial tcp: lookup invisiblemoon.xyz: no such host" fetcher=DirectKeyFetcher Aug 16 17:27:08 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:27:08.692644254Z" level=warning msg="Failed to retrieve any keys" context=missing fetcher=DirectKeyFetcher Aug 16 17:27:08 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:27:08.692471749Z" level=error msg="Failed to fetch key for server" context=missing error="Post \"matrix://decryptology.net/_matrix/key/v2/query\": dial tcp 104.21.52.154:8448: i/o timeout" fetcher=DirectKeyFetcher Aug 16 17:26:45 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:26:45.968449825Z" level=warning msg="Failed to retrieve any keys" context=missing fetcher=DirectKeyFetcher Aug 16 17:26:45 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:26:45.968363645Z" level=error msg="Failed to fetch key for server" context=missing error="Post \"matrix://amba.cloud/_matrix/key/v2/query\": dial tcp 178.154.227.247:8448: i/o timeout" fetcher=DirectKeyFetcher Aug 16 17:26:25 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:26:25.285394219Z" level=warning msg="Failed to retrieve any keys" context=missing fetcher=DirectKeyFetcher Aug 16 17:26:25 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:26:25.285237416Z" level=error msg="Failed to fetch key for server" context=missing error="Post \"matrix://matrix.adawesome.tech/_matrix/key/v2/query\": dial tcp 68.72.223.216:8448: i/o timeout" fetcher=DirectKeyFetcher Aug 16 17:26:03 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:26:03.913695554Z" level=warning msg="Failed to retrieve any keys" context=missing fetcher=DirectKeyFetcher Aug 16 17:26:03 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:26:03.913010959Z" level=error msg="Failed to fetch key for server" context=missing error="Post \"matrix://matrix.wigoftime.net/_matrix/key/v2/query\": dial tcp: lookup matrix.wigoftime.net: i/o timeout" fetcher=DirectKeyFetcher Aug 16 17:25:48 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:25:48.586647668Z" level=warning msg="Failed to retrieve any keys" context=missing fetcher=DirectKeyFetcher Aug 16 17:25:48 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:25:48.586558218Z" level=error msg="Failed to fetch key for server" context=missing error="Post \"matrix://applied.at/_matrix/key/v2/query\": x509: certificate is valid for applied.at, autodiscover.applied.at, exchange.applied.at, outlook.applied.at, portal.applied.at, va12.applied.at, www.applied.at, not matrix.applied.at" fetcher=DirectKeyFetcher Aug 16 17:25:45 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:25:45.964519296Z" level=warning msg="Failed to retrieve any keys" context=missing fetcher=DirectKeyFetcher Aug 16 17:25:45 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:25:45.964149266Z" level=error msg="Failed to fetch key for server" context=missing error="Post \"matrix://shipp.dev/_matrix/key/v2/query\": dial tcp 172.67.145.187:8448: i/o timeout" fetcher=DirectKeyFetcher Aug 16 17:25:25 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:25:25.620867733Z" level=warning msg="Failed to retrieve any keys" context=missing fetcher=DirectKeyFetcher Aug 16 17:25:25 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:25:25.620739752Z" level=error msg="Failed to fetch key for server" context=missing error="Post \"matrix://matrix.declicnumerique.be/_matrix/key/v2/query\": context deadline exceeded" fetcher=DirectKeyFetcher Aug 16 17:25:14 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:25:14.949702101Z" level=info msg="User joined room successfully" room_id="#pine64germany:matrix.org" servers="[matrix.org]" user_id="@me:example.com" Aug 16 17:25:09 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:25:09.463594500Z" level=info msg=DeviceListUpdater.Update context=missing deleted=false device_id=NZRKGQNEWL display_name="" prev_ids="[4971658125]" prev_ids_exist=false stream_id=4971686209 user_id="@jwh4j4ez25q:matrix.org" Aug 16 17:25:02 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:25:02.611938094Z" level=info msg="Joined federated room with hosts" hosts="[{$FCibUYM_5VU61qn0BEj3kShWE_eWlEgn7c83PuZ4ivs matrix.org} {$Ttdl_zYKkjhfuHzr2UVrQ8A61YE9oyGMqPluzEajXWo matrix.org} {$lL965n5cwqLL8ysD3-LfNa6KmUhUg0pRZ4od03DLaXs riotchat.de} {$uwb20WqtFC-phKjs13rOGlkYrEbp05cHfiLpSj4lrbQ matrix.org} {$L2Qy7iCnd6zjcoPW_Esk0SevvEVq20v6zRI2fH8yrso matrix.org} {$vjV2qJv-V9DJmxomVWNUFpgi47HpvGwzSEJAvAA3Rzg matrix.org} {$iAgkXlCfQy49dORd0T-iQ5X3X-uZ3KcUpXMhgU0PxoQ matrix.org} {$-KX4tTMWghrxAFwPVcdl69ltLnGMeNhTyRCERJj1fc8 matrix.org} {$ureIQWWfFgTP-We2YWE46Oe6eYBMRXz65ov0onyZ4PM matrix.org} {$pvG3acHPEoT5ZD3wKIZmZ42f1e8XnQpxP_mEYJgqems matrix.org} {$CU2q41joYyEQIst2W07mx3kK4bZzE4olTK3xts-6kPA sibnsk.net}]" room="!oPZStMrnhOBXXxTfZz:matrix.org" Aug 16 17:25:02 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:25:02.608712596Z" level=warning msg="Discarding 4 auth/state event(s) due to invalid signatures" context=missing Aug 16 17:25:00 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:25:00.132548887Z" level=warning msg="Signature validation failed for event \"$0eGcn3yWfTpaIL7FxRunRGiQNpr87fGohL72BxptVqc\"" error="gomatrixserverlib: could not download key for \"32mb.de\"" Aug 16 17:25:00 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:25:00.132509373Z" level=warning msg="Signature validation failed for event \"$ifND9abpUgJFXiGAfCZwEeJRzva1SzaOiVUwF7jC-WE\"" error="gomatrixserverlib: could not download key for \"32mb.de\"" Aug 16 17:25:00 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:25:00.132459859Z" level=warning msg="Signature validation failed for event \"$bPt4m5FER_LOlNwysAxR0Pjs_E6qv3u3vyODKpwkAl4\"" error="gomatrixserverlib: could not download key for \"32mb.de\"" Aug 16 17:25:00 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:25:00.132401196Z" level=warning msg="Signature validation failed for event \"$b225MQeZNjxWn4ZgKImTkv108sMJxWq86e8RE41vIZY\"" error="gomatrixserverlib: could not download key for \"32mb.de\"" Aug 16 17:24:54 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:24:54.123646366Z" level=info msg="Checking event signatures for 60 events of room state" context=missing Aug 16 17:24:53 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:24:53.879159001Z" level=warning msg="Failed to retrieve any keys" context=missing fetcher=DirectKeyFetcher Aug 16 17:24:53 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:24:53.879001985Z" level=error msg="Failed to fetch key for server" context=missing error="Post \"matrix://searx.monster/_matrix/key/v2/query\": dial tcp: lookup searx.monster: i/o timeout" fetcher=DirectKeyFetcher Aug 16 17:24:50 nixos dendrite-monolith-server[51730]: time="2022-08-16T21:24:50.680340627Z" level=info msg="User requested to room join" room_id="#pine64germany:matrix.org" servers="[matrix.org]" user_id="@me:example.com" ```

matrix-org / dendrite

can't join medium sized federated rooms #2153

Background information

Description

Steps to reproduce

502 Bad Gateway