search

element-hq / element-ios

A glossy Matrix collaboration client for iOS
https://element.io
GNU Affero General Public License v3.0
1.73k stars 499 forks source link

VOIP Call on working on different network connection. #3059

Open ghost opened 4 years ago

ghost commented 4 years ago

We are facing issues in VOIP(Audio & Video) calls. Its working on same wifi network but always stuck in call connecting when switch to different network or 4G connection.

We have used the latest Matrix SDK in our project. Please support us to resolve this issue.

Wodann commented 3 years ago

I'm experiencing this issue too when calling from or being called on an Android device (on WiFi) to iOS device (on 4G).

Might be related to https://github.com/vector-im/element-android/issues/2255

johansmitsnl commented 3 years ago

I experience the same issue but sometimes it works but most of the times not. Element web or desktop with iOS works fine. The connection is not relevant in my case. Both WiFi or 4g or a combination of it keeps it the same. Desktop and web are both on a local ISP with utp connection.

johansmitsnl commented 3 years ago

I have to correct my image. On my side it was a missing a good working coturn server. Once configured I have always a good and direct connection. From the synapse admin channel I also found this website to test your turn setup: https://test.voip.librepush.net/

HeikoBoettger commented 3 years ago

I currently see similar issues although the turnserver is according to https://webrtc.github.io/samples/src/content/peerconnection/trickle-ice/ working as expected. One thing I discovered is that the test seems to only do some basic connection tests. For more complete testing you need at least six clients, which means two of the following clients:

client 1: ipv4 from a known server network range (because of server-relay option in coturn) client 2: ipv4 from a typicall clent isp network (because of server-relay option) client 3: ipv6 client

Which results in at least into the following tests: client 1a <-> client1b client 1a <-> client2b client 1a <-> client3b

client 2a <-> client1b client 2a <-> client2b client 2a <-> client3b

client 3a <-> client1b client 3a <-> client2b client 3a <-> client3b

Not knowing about the filters on coturn, there might be even the need to test whether clients behind a Carrier-grade NAT (RFC 6264) are working. The critical issue is that software such as element might not use the turn-server at all in case the clients are inside the same network or firewall hole-punching is working.

Is there a more complete testing-service on the web?

J4gQBqqR commented 2 years ago

Scenario 1:

Device 1 5G/LTE <-----> router <-----> Synapse Server
Device 2 5G/LTE <--|

Calling between Device 1 and Device 2 works.

Scenario 2:

router <-----> Synapse Server
          |--> Device 1 (WiFi)
          |--> Device 2 (WiFi)

Calling between Device 1 and Device 2 works.

Scenario 3:

Device 1 5G/LTE <-----> router <-----> Synapse Server
                                  |--> Device 2 (WiFi)

Synapse Server and Device 2 are under the same LAN, e.g. Server is 10.0.0.2, Device 2 is 10.0.0.5 Calling between Device 1 and Device 2 does not work. The App always show "Call Connecting..."

Scenario 4:

router <-----> Synapse Server
          |--> Device 1 (WiFi) behind Windscribe VPN
          |--> Device 2 (WiFi)

Calling between Device 1 and Device 2 does not work. The App always show "Call Connecting..."

Scenario 5:

router <-----> Synapse Server
          |--> Device 1 (WiFi) behind Windscribe VPN
          |--> Device 2 (WiFi) behind Windscribe VPN

Calling between Device 1 and Device 2 does not work. The App always show "Call Connecting..."

Scenario 6:

Device 1 5G/LTE behind Windscribe VPN <-----> router <-----> Synapse Server
Device 2 5G/LTE behind Windscribe VPN <--|

Calling between Device 1 and Device 2 does not work. The App always show "Call Connecting..."

Scenario 7:

Device 1 5G/LTE behind Windscribe VPN <-----> router <-----> Synapse Server
                      Device 2 5G/LTE <--|

Calling between Device 1 and Device 2 does not work. The App always show "Call Connecting..."

More info: In my homeserver.yaml:

turn_uris: [ "turns:staticauth.openrelay.metered.ca:443" ]
turn_shared_secret: "openrelayprojectsecret"
turn_allow_guests: true

I am using TURN server from https://www.metered.ca/tools/openrelay/ I tried to host my own coturn server, the result is exactly the same as above.

J4gQBqqR commented 2 years ago 
2022-11-20 11:08:39,055 - synapse.access.http.8008 - 460 - INFO - GET-960 - ::ffff:172.19.0.13 - 8008 - {@telegrambot:xxxx.com} Processed request: 30.002sec/0.000sec (0.002sec, 0.000sec) (0.000sec/0.001sec/1) 197B 200 "GET /_matrix/client/v3/sync?timeout=30000&since=s1223_8346_0_315_396_4_145_79_0&filter=0&set_presence=online HTTP/1.1" "mautrix-telegram/0.12.1+dev.5558fc71 mautrix-python/0.18.6 aiohttp/3.8.1 Python/3.10.5" [0 dbevts]
2022-11-20 11:08:39,478 - synapse.access.http.8008 - 460 - INFO - GET-973 - ::ffff:172.19.0.13 - 8008 - {@telegrambot:xxxx.com} Processed request: 0.008sec/0.001sec (0.006sec, 0.000sec) (0.001sec/0.002sec/1) 360B 200 "GET /_matrix/client/v3/sync?timeout=30000&since=s1223_8346_0_315_396_4_145_79_0&filter=0&set_presence=online HTTP/1.1" "mautrix-telegram/0.12.1+dev.5558fc71 mautrix-python/0.18.6 aiohttp/3.8.1 Python/3.10.5" [0 dbevts]
2022-11-20 11:08:41,282 - synapse.access.http.8008 - 460 - INFO - GET-975 - 10.0.0.161 - 8008 - {@ssss:xxxx.com} Processed request: 0.001sec/0.000sec (0.001sec, 0.000sec) (0.000sec/0.000sec/0) 155B 200 "GET /_matrix/client/r0/voip/turnServer HTTP/1.1" "Element/1.5.7 (Xiaomi 21051182C; Android 13; TD1A.221105.003; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [0 dbevts]
2022-11-20 11:08:42,016 - synapse.access.http.8008 - 460 - INFO - GET-961 - ::ffff:172.19.0.15 - 8008 - {@signalbot:xxxx.com} Processed request: 30.003sec/0.000sec (0.002sec, 0.000sec) (0.001sec/0.002sec/1) 197B 200 "GET /_matrix/client/v3/sync?timeout=30000&since=s1223_8346_0_315_396_4_145_79_0&filter=0&set_presence=online HTTP/1.1" "mautrix-signal/0.4.1+dev.3cbdac88 mautrix-python/0.18.6 aiohttp/3.8.1 Python/3.10.5" [0 dbevts]
2022-11-20 11:08:42,036 - synapse.metrics._gc - 120 - INFO - sentinel - Collecting gc 1
2022-11-20 11:08:42,044 - synapse.access.http.8008 - 460 - INFO - GET-977 - ::ffff:172.19.0.15 - 8008 - {@signalbot:xxxx.com} Processed request: 0.002sec/0.000sec (0.000sec, 0.000sec) (0.000sec/0.000sec/1) 495B 200 "GET /_matrix/client/v3/sync?timeout=30000&since=s1223_8346_0_315_396_4_145_79_0&filter=0&set_presence=online HTTP/1.1" "mautrix-signal/0.4.1+dev.3cbdac88 mautrix-python/0.18.6 aiohttp/3.8.1 Python/3.10.5" [0 dbevts]
2022-11-20 11:08:42,816 - synapse.access.http.8008 - 460 - INFO - PUT-976 - 10.0.0.161 - 8008 - {@ssss:xxxx.com} Processed request: 1.215sec/0.010sec (0.012sec, 0.010sec) (0.002sec/0.273sec/5) 59B 200 "PUT /_matrix/client/r0/rooms/!MHURLalqKLvNdeLuoC:xxxx.com/send/m.room.encrypted/$local.1c9a3162-c1ea-49a3-94af-a12dcc9caabe HTTP/1.1" "Element/1.5.7 (Xiaomi 21051182C; Android 13; TD1A.221105.003; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [0 dbevts]
2022-11-20 11:08:43,005 - synapse.federation.sender - 457 - INFO - process_event_queue_for_federation-12 - Unexpectedly did not have cached prev group for $0km9scZH70nrj8cNtlDsi49H1_vkpcg-q1YjqsS_yMs
2022-11-20 11:08:43,009 - synapse.access.http.8008 - 460 - INFO - GET-971 - 172.58.249.6 - 8008 - {@yyyy:xxxx.com} Processed request: 16.603sec/0.000sec (0.009sec, 0.001sec) (0.007sec/0.012sec/4) 2762B 200 "GET /_matrix/client/r0/sync?filter=1&set_presence=online&timeout=30000&since=s1223_8346_0_315_396_4_145_79_0 HTTP/1.1" "Element/1.5.7 (motorola XT2125-4; Android 12; S1RN32.55-16-2; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [1 dbevts]
2022-11-20 11:08:43,024 - synapse.access.http.8008 - 460 - INFO - GET-967 - 10.0.0.1 - 8008 - {@ssss:xxxx.com} Processed request: 25.984sec/0.003sec (0.003sec, 0.001sec) (0.013sec/0.030sec/4) 2264B 200 "GET /_matrix/client/r0/sync?filter=0&set_presence=online&timeout=30000&since=s1223_8346_0_315_396_4_145_79_0 HTTP/1.1" "Element/1.5.7 (Xiaomi 21051182C; Android 13; TD1A.221105.003; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [0 dbevts]
2022-11-20 11:08:43,025 - synapse.access.http.8008 - 460 - INFO - GET-962 - 10.0.0.6 - 8008 - {@ssss:xxxx.com} Processed request: 28.762sec/0.002sec (0.002sec, 0.000sec) (0.014sec/0.013sec/3) 2201B 200 "GET /_matrix/client/r0/sync?filter=0&set_presence=online&timeout=30000&since=s1223_8346_0_315_396_4_145_79_0 HTTP/1.1" "Element/1.5.4 (Xiaomi 2112123AC; Android 12; SKQ1.211230.001 test-keys; Flavour FDroid; MatrixAndroidSdk2 1.5.4)" [0 dbevts]
2022-11-20 11:08:43,262 - synapse.access.http.8008 - 460 - INFO - PUT-979 - 10.0.0.161 - 8008 - {@ssss:xxxx.com} Processed request: 0.227sec/0.007sec (0.004sec, 0.001sec) (0.002sec/0.048sec/5) 59B 200 "PUT /_matrix/client/r0/rooms/!MHURLalqKLvNdeLuoC:xxxx.com/send/m.room.encrypted/$local.0e2ed8d1-ac7e-4b87-a980-42df95cc4162 HTTP/1.1" "Element/1.5.7 (Xiaomi 21051182C; Android 13; TD1A.221105.003; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [0 dbevts]
2022-11-20 11:08:43,266 - synapse.federation.sender - 457 - INFO - process_event_queue_for_federation-13 - Unexpectedly did not have cached prev group for $WxnInQ3rIq1vj3LbHk_cNfWAa-3rld-HiPVLDB_FPnM
2022-11-20 11:08:43,277 - synapse.access.http.8008 - 460 - INFO - GET-980 - 10.0.0.1 - 8008 - {@ssss:xxxx.com} Processed request: 0.063sec/0.001sec (0.004sec, 0.002sec) (0.006sec/0.013sec/4) 3352B 200 "GET /_matrix/client/r0/sync?filter=0&set_presence=online&timeout=30000&since=s1224_8353_0_315_396_4_145_79_0 HTTP/1.1" "Element/1.5.7 (Xiaomi 21051182C; Android 13; TD1A.221105.003; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [1 dbevts]
2022-11-20 11:08:43,289 - synapse.access.http.8008 - 460 - INFO - GET-981 - 10.0.0.6 - 8008 - {@ssss:xxxx.com} Processed request: 0.006sec/0.001sec (0.002sec, 0.000sec) (0.000sec/0.001sec/2) 3289B 200 "GET /_matrix/client/r0/sync?filter=0&set_presence=online&timeout=30000&since=s1224_8353_0_315_396_4_145_79_0 HTTP/1.1" "Element/1.5.4 (Xiaomi 2112123AC; Android 12; SKQ1.211230.001 test-keys; Flavour FDroid; MatrixAndroidSdk2 1.5.4)" [0 dbevts]
2022-11-20 11:08:43,398 - synapse.rest.media.v1.media_repository - 476 - WARNING - GET-982 - Not retrying destination 'chat.xxxx.com'
2022-11-20 11:08:43,400 - synapse.http.server - 108 - INFO - GET-982 - <XForwardedForRequest at 0x7fddb6e72d90 method='GET' uri='/_matrix/media/r0/thumbnail/chat.xxxx.com/cAZsTMsbsIbaZBzZoCVefESp?width=250&height=250&method=scale' clientproto='HTTP/1.1' site='8008'> SynapseError: 502 - Failed to fetch remote media
2022-11-20 11:08:43,403 - synapse.access.http.8008 - 460 - INFO - GET-982 - 172.58.249.6 - 8008 - {None} Processed request: 0.010sec/0.002sec (0.000sec, 0.003sec) (0.001sec/0.004sec/1) 62B 502 "GET /_matrix/media/r0/thumbnail/chat.xxxx.com/cAZsTMsbsIbaZBzZoCVefESp?width=250&height=250&method=scale HTTP/1.1" "Element/1.5.7 (motorola XT2125-4; Android 12; S1RN32.55-16-2; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [0 dbevts]
2022-11-20 11:08:43,426 - synapse.access.http.8008 - 460 - INFO - GET-983 - 172.58.249.6 - 8008 - {@yyyy:xxxx.com} Processed request: 0.031sec/0.001sec (0.006sec, 0.002sec) (0.003sec/0.015sec/3) 3289B 200 "GET /_matrix/client/r0/sync?filter=1&set_presence=online&timeout=30000&since=s1224_8353_0_315_396_4_145_79_0 HTTP/1.1" "Element/1.5.7 (motorola XT2125-4; Android 12; S1RN32.55-16-2; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [0 dbevts]
2022-11-20 11:08:43,461 - synapse.push.httppusher - 225 - INFO - httppush.process-15 - Processing 1 unprocessed push actions for @yyyy:xxxx.com/im.vector.app.android/fgQCfmGEQBaZEM7NzVXJAl:APA91bGDP93cyMrollz6TZEx02CuYE_d1bUkK6uIyjAhcjOKy_DgxMgpvij60a27ExYzLQ3v2KASBgy8bC1Tx3s01TBNPPOtL6fI_lxk8Ow0gqB3Pbb4q8ZYmH2BNFP8KlM1kaP8mJ1A starting at stream_ordering 1223
2022-11-20 11:08:43,697 - synapse.access.http.8008 - 460 - INFO - PUT-984 - 10.0.0.1 - 8008 - {@ssss:xxxx.com} Processed request: 0.233sec/0.006sec (0.007sec, 0.001sec) (0.003sec/0.041sec/5) 59B 200 "PUT /_matrix/client/r0/rooms/!MHURLalqKLvNdeLuoC:xxxx.com/send/m.room.encrypted/$local.86a723b4-d54f-4c25-99ba-e443dffbdfb9 HTTP/1.1" "Element/1.5.7 (Xiaomi 21051182C; Android 13; TD1A.221105.003; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [0 dbevts]
2022-11-20 11:08:43,725 - synapse.federation.sender - 457 - INFO - process_event_queue_for_federation-14 - Unexpectedly did not have cached prev group for $haUdsjjz8Ocnx8slr9b8JCb35RAfEO7_QSpsNa_1HsU
2022-11-20 11:08:43,737 - synapse.access.http.8008 - 460 - INFO - GET-985 - 10.0.0.161 - 8008 - {@ssss:xxxx.com} Processed request: 0.240sec/0.002sec (0.010sec, 0.001sec) (0.015sec/0.020sec/4) 1773B 200 "GET /_matrix/client/r0/sync?filter=0&set_presence=online&timeout=30000&since=s1225_8353_0_315_396_4_145_79_0 HTTP/1.1" "Element/1.5.7 (Xiaomi 21051182C; Android 13; TD1A.221105.003; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [0 dbevts]
2022-11-20 11:08:43,738 - synapse.access.http.8008 - 460 - INFO - GET-986 - 10.0.0.6 - 8008 - {@ssss:xxxx.com} Processed request: 0.221sec/0.002sec (0.004sec, 0.001sec) (0.010sec/0.010sec/3) 1710B 200 "GET /_matrix/client/r0/sync?filter=0&set_presence=online&timeout=30000&since=s1225_8353_0_315_396_4_145_79_0 HTTP/1.1" "Element/1.5.4 (Xiaomi 2112123AC; Android 12; SKQ1.211230.001 test-keys; Flavour FDroid; MatrixAndroidSdk2 1.5.4)" [1 dbevts]
2022-11-20 11:08:43,738 - synapse.access.http.8008 - 460 - INFO - GET-987 - 172.58.249.6 - 8008 - {@yyyy:xxxx.com} Processed request: 0.037sec/0.002sec (0.010sec, 0.000sec) (0.012sec/0.009sec/3) 1710B 200 "GET /_matrix/client/r0/sync?filter=1&set_presence=online&timeout=30000&since=s1225_8353_0_315_396_4_145_79_0 HTTP/1.1" "Element/1.5.7 (motorola XT2125-4; Android 12; S1RN32.55-16-2; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [0 dbevts]
2022-11-20 11:08:43,763 - synapse.rest.media.v1.media_repository - 476 - WARNING - GET-988 - Not retrying destination 'chat.xxxx.com'
2022-11-20 11:08:43,764 - synapse.http.server - 108 - INFO - GET-988 - <XForwardedForRequest at 0x7fddb6170610 method='GET' uri='/_matrix/media/r0/thumbnail/chat.xxxx.com/cAZsTMsbsIbaZBzZoCVefESp?width=250&height=250&method=scale' clientproto='HTTP/1.1' site='8008'> SynapseError: 502 - Failed to fetch remote media
2022-11-20 11:08:43,766 - synapse.access.http.8008 - 460 - INFO - GET-988 - 172.58.249.6 - 8008 - {None} Processed request: 0.015sec/0.002sec (0.002sec, 0.001sec) (0.000sec/0.001sec/1) 62B 502 "GET /_matrix/media/r0/thumbnail/chat.xxxx.com/cAZsTMsbsIbaZBzZoCVefESp?width=250&height=250&method=scale HTTP/1.1" "Element/1.5.7 (motorola XT2125-4; Android 12; S1RN32.55-16-2; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [0 dbevts]
2022-11-20 11:08:44,125 - synapse.rest.media.v1.media_repository - 476 - WARNING - GET-992 - Not retrying destination 'chat.xxxx.com'
2022-11-20 11:08:44,126 - synapse.http.server - 108 - INFO - GET-992 - <XForwardedForRequest at 0x7fddb6149eb0 method='GET' uri='/_matrix/media/r0/thumbnail/chat.xxxx.com/cAZsTMsbsIbaZBzZoCVefESp?width=250&height=250&method=scale' clientproto='HTTP/1.1' site='8008'> SynapseError: 502 - Failed to fetch remote media
2022-11-20 11:08:44,128 - synapse.access.http.8008 - 460 - INFO - GET-992 - 172.58.249.6 - 8008 - {None} Processed request: 0.006sec/0.001sec (0.003sec, 0.000sec) (0.001sec/0.002sec/1) 62B 502 "GET /_matrix/media/r0/thumbnail/chat.xxxx.com/cAZsTMsbsIbaZBzZoCVefESp?width=250&height=250&method=scale HTTP/1.1" "Element/1.5.7 (motorola XT2125-4; Android 12; S1RN32.55-16-2; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [0 dbevts]
2022-11-20 11:08:44,370 - synapse.http.client - 460 - INFO - httppush.process-15 - Received response to POST https://matrix.org/_matrix/push/v1/notify: 200
2022-11-20 11:08:44,447 - synapse.push.httppusher - 225 - INFO - httppush.process-15 - Processing 2 unprocessed push actions for @yyyy:xxxx.com/im.vector.app.android/fgQCfmGEQBaZEM7NzVXJAl:APA91bGDP93cyMrollz6TZEx02CuYE_d1bUkK6uIyjAhcjOKy_DgxMgpvij60a27ExYzLQ3v2KASBgy8bC1Tx3s01TBNPPOtL6fI_lxk8Ow0gqB3Pbb4q8ZYmH2BNFP8KlM1kaP8mJ1A starting at stream_ordering 1224
2022-11-20 11:08:44,457 - synapse.rest.media.v1.media_repository - 476 - WARNING - GET-994 - Not retrying destination 'chat.xxxx.com'
2022-11-20 11:08:44,458 - synapse.http.server - 108 - INFO - GET-994 - <XForwardedForRequest at 0x7fddb6e8b310 method='GET' uri='/_matrix/media/r0/thumbnail/chat.xxxx.com/cAZsTMsbsIbaZBzZoCVefESp?width=250&height=250&method=scale' clientproto='HTTP/1.1' site='8008'> SynapseError: 502 - Failed to fetch remote media
2022-11-20 11:08:44,461 - synapse.access.http.8008 - 460 - INFO - GET-994 - 172.58.249.6 - 8008 - {None} Processed request: 0.007sec/0.002sec (0.003sec, 0.000sec) (0.001sec/0.001sec/1) 62B 502 "GET /_matrix/media/r0/thumbnail/chat.xxxx.com/cAZsTMsbsIbaZBzZoCVefESp?width=250&height=250&method=scale HTTP/1.1" "Element/1.5.7 (motorola XT2125-4; Android 12; S1RN32.55-16-2; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [0 dbevts]
2022-11-20 11:08:44,549 - synapse.access.http.8008 - 460 - INFO - POST-993 - 172.58.249.6 - 8008 - {@yyyy:xxxx.com} Processed request: 0.286sec/0.001sec (0.008sec, 0.001sec) (0.007sec/0.260sec/5) 2B 200 "POST /_matrix/client/r0/rooms/!MHURLalqKLvNdeLuoC:xxxx.com/receipt/m.read/$haUdsjjz8Ocnx8slr9b8JCb35RAfEO7_QSpsNa_1HsU HTTP/1.1" "Element/1.5.7 (motorola XT2125-4; Android 12; S1RN32.55-16-2; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [0 dbevts]
2022-11-20 11:08:44,570 - synapse.access.http.8008 - 460 - INFO - GET-990 - 172.58.249.6 - 8008 - {@yyyy:xxxx.com} Processed request: 0.605sec/0.004sec (0.006sec, 0.001sec) (0.003sec/0.008sec/2) 680B 200 "GET /_matrix/client/r0/sync?filter=1&set_presence=online&timeout=30000&since=s1226_8353_0_315_396_4_145_79_0 HTTP/1.1" "Element/1.5.7 (motorola XT2125-4; Android 12; S1RN32.55-16-2; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [0 dbevts]
2022-11-20 11:08:44,580 - synapse.access.http.8008 - 460 - INFO - GET-991 - 10.0.0.6 - 8008 - {@ssss:xxxx.com} Processed request: 0.560sec/0.002sec (0.006sec, 0.002sec) (0.010sec/0.012sec/3) 680B 200 "GET /_matrix/client/r0/sync?filter=0&set_presence=online&timeout=30000&since=s1226_8353_0_315_396_4_145_79_0 HTTP/1.1" "Element/1.5.4 (Xiaomi 2112123AC; Android 12; SKQ1.211230.001 test-keys; Flavour FDroid; MatrixAndroidSdk2 1.5.4)" [0 dbevts]
2022-11-20 11:08:44,581 - synapse.access.http.8008 - 460 - INFO - GET-989 - 10.0.0.1 - 8008 - {@ssss:xxxx.com} Processed request: 0.646sec/0.002sec (0.002sec, 0.000sec) (0.006sec/0.024sec/3) 680B 200 "GET /_matrix/client/r0/sync?filter=0&set_presence=online&timeout=30000&since=s1226_8353_0_315_396_4_145_79_0 HTTP/1.1" "Element/1.5.7 (Xiaomi 21051182C; Android 13; TD1A.221105.003; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [0 dbevts]
2022-11-20 11:08:44,735 - synapse.rest.media.v1.media_repository - 476 - WARNING - GET-995 - Not retrying destination 'chat.xxxx.com'
2022-11-20 11:08:44,736 - synapse.http.server - 108 - INFO - GET-995 - <XForwardedForRequest at 0x7fddb6078070 method='GET' uri='/_matrix/media/r0/thumbnail/chat.xxxx.com/cAZsTMsbsIbaZBzZoCVefESp?width=250&height=250&method=scale' clientproto='HTTP/1.1' site='8008'> SynapseError: 502 - Failed to fetch remote media
2022-11-20 11:08:44,739 - synapse.access.http.8008 - 460 - INFO - GET-995 - 172.58.249.6 - 8008 - {None} Processed request: 0.005sec/0.002sec (0.001sec, 0.000sec) (0.004sec/0.000sec/1) 62B 502 "GET /_matrix/media/r0/thumbnail/chat.xxxx.com/cAZsTMsbsIbaZBzZoCVefESp?width=250&height=250&method=scale HTTP/1.1" "Element/1.5.7 (motorola XT2125-4; Android 12; S1RN32.55-16-2; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [0 dbevts]
2022-11-20 11:08:44,749 - synapse.http.client - 460 - INFO - httppush.process-15 - Received response to POST https://matrix.org/_matrix/push/v1/notify: 200
2022-11-20 11:08:44,915 - synapse.http.client - 460 - INFO - httppush.process-15 - Received response to POST https://matrix.org/_matrix/push/v1/notify: 200
2022-11-20 11:08:45,563 - synapse.rest.media.v1.media_repository - 476 - WARNING - GET-1000 - Not retrying destination 'chat.xxxx.com'
2022-11-20 11:08:45,564 - synapse.http.server - 108 - INFO - GET-1000 - <XForwardedForRequest at 0x7fddb61b74c0 method='GET' uri='/_matrix/media/r0/thumbnail/chat.xxxx.com/cAZsTMsbsIbaZBzZoCVefESp?width=250&height=250&method=scale' clientproto='HTTP/1.1' site='8008'> SynapseError: 502 - Failed to fetch remote media
2022-11-20 11:08:45,566 - synapse.access.http.8008 - 460 - INFO - GET-1000 - 172.58.249.6 - 8008 - {None} Processed request: 0.006sec/0.001sec (0.001sec, 0.001sec) (0.001sec/0.001sec/1) 62B 502 "GET /_matrix/media/r0/thumbnail/chat.xxxx.com/cAZsTMsbsIbaZBzZoCVefESp?width=250&height=250&method=scale HTTP/1.1" "Element/1.5.7 (motorola XT2125-4; Android 12; S1RN32.55-16-2; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [0 dbevts]
2022-11-20 11:08:45,738 - synapse.access.http.8008 - 460 - INFO - GET-963 - ::ffff:172.19.0.14 - 8008 - {@qqbot:xxxx.com} Processed request: 30.004sec/0.001sec (0.003sec, 0.000sec) (0.001sec/0.002sec/1) 197B 200 "GET /_matrix/client/v3/sync?filter=0&since=s1223_8346_0_315_396_4_145_79_0&timeout=30000 HTTP/1.1" "matrix-qq/0.0.10+dev.unknown mautrix-go/v0.12.2" [0 dbevts]
2022-11-20 11:08:45,895 - synapse.rest.media.v1.media_repository - 476 - WARNING - GET-1001 - Not retrying destination 'chat.xxxx.com'
2022-11-20 11:08:45,896 - synapse.http.server - 108 - INFO - GET-1001 - <XForwardedForRequest at 0x7fddb6ed4e20 method='GET' uri='/_matrix/media/r0/thumbnail/chat.xxxx.com/cAZsTMsbsIbaZBzZoCVefESp?width=250&height=250&method=scale' clientproto='HTTP/1.1' site='8008'> SynapseError: 502 - Failed to fetch remote media
2022-11-20 11:08:45,898 - synapse.access.http.8008 - 460 - INFO - GET-1001 - 172.58.249.6 - 8008 - {None} Processed request: 0.005sec/0.001sec (0.003sec, 0.000sec) (0.001sec/0.001sec/1) 62B 502 "GET /_matrix/media/r0/thumbnail/chat.xxxx.com/cAZsTMsbsIbaZBzZoCVefESp?width=250&height=250&method=scale HTTP/1.1" "Element/1.5.7 (motorola XT2125-4; Android 12; S1RN32.55-16-2; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [0 dbevts]
2022-11-20 11:08:45,911 - synapse.access.http.8008 - 460 - INFO - GET-964 - ::ffff:172.19.0.12 - 8008 - {@whatsappbot:xxxx.com} Processed request: 30.004sec/0.001sec (0.003sec, 0.000sec) (0.001sec/0.002sec/1) 197B 200 "GET /_matrix/client/v3/sync?filter=0&since=s1223_8346_0_315_396_4_145_79_0&timeout=30000 HTTP/1.1" "mautrix-whatsapp/0.7.2 mautrix-go/v0.12.3" [0 dbevts]
2022-11-20 11:08:45,921 - synapse.access.http.8008 - 460 - INFO - GET-965 - ::ffff:172.19.0.11 - 8008 - {@discordbot:xxxx.com} Processed request: 30.003sec/0.001sec (0.002sec, 0.000sec) (0.001sec/0.002sec/1) 197B 200 "GET /_matrix/client/v3/sync?filter=0&since=s1223_8346_0_315_396_4_145_79_0&timeout=30000 HTTP/1.1" "mautrix-discord/0.1.0+dev.f53975cc mautrix-go/v0.12.2" [0 dbevts]
2022-11-20 11:08:46,272 - synapse.access.http.8008 - 460 - INFO - GET-1002 - ::ffff:172.19.0.12 - 8008 - {@whatsappbot:xxxx.com} Processed request: 0.011sec/0.002sec (0.002sec, 0.001sec) (0.001sec/0.004sec/1) 496B 200 "GET /_matrix/client/v3/sync?filter=0&since=s1223_8346_0_315_396_4_145_79_0&timeout=30000 HTTP/1.1" "mautrix-whatsapp/0.7.2 mautrix-go/v0.12.3" [0 dbevts]
2022-11-20 11:08:46,276 - synapse.access.http.8008 - 460 - INFO - GET-1003 - ::ffff:172.19.0.14 - 8008 - {@qqbot:xxxx.com} Processed request: 0.012sec/0.001sec (0.004sec, 0.000sec) (0.001sec/0.005sec/1) 491B 200 "GET /_matrix/client/v3/sync?filter=0&since=s1223_8346_0_315_396_4_145_79_0&timeout=30000 HTTP/1.1" "matrix-qq/0.0.10+dev.unknown mautrix-go/v0.12.2" [0 dbevts]
2022-11-20 11:08:46,310 - synapse.access.http.8008 - 460 - INFO - GET-1004 - ::ffff:172.19.0.11 - 8008 - {@discordbot:xxxx.com} Processed request: 0.009sec/0.001sec (0.001sec, 0.000sec) (0.001sec/0.002sec/1) 495B 200 "GET /_matrix/client/v3/sync?filter=0&since=s1223_8346_0_315_396_4_145_79_0&timeout=30000 HTTP/1.1" "mautrix-discord/0.1.0+dev.f53975cc mautrix-go/v0.12.2" [0 dbevts]
2022-11-20 11:08:46,511 - synapse.access.http.8008 - 460 - INFO - PUT-999 - 172.58.249.6 - 8008 - {@yyyy:xxxx.com} Processed request: 0.945sec/0.009sec (0.010sec, 0.004sec) (0.003sec/0.622sec/5) 59B 200 "PUT /_matrix/client/r0/rooms/!MHURLalqKLvNdeLuoC:xxxx.com/send/m.room.encrypted/$local.9aad9c2f-2188-4008-8c9f-3c961d47ff53 HTTP/1.1" "Element/1.5.7 (motorola XT2125-4; Android 12; S1RN32.55-16-2; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [0 dbevts]
2022-11-20 11:08:46,529 - synapse.push.httppusher - 225 - INFO - httppush.process-16 - Processing 1 unprocessed push actions for @ssss:xxxx.com/im.vector.app.android/c5ExiLQzRU2ZU1TUmr-Ztb:APA91bEvl0jRCcDMoyNlEhcEIzV671hwNf5FNd5VTyutaGibNylPorDNloTwQ0V9aNN3mMgEZRdoAaKWLazHxtiaazGYgFnatojmFwXtBBiBa4WyxG0wSMEDgaXGjQU7geXx5dmjPWF5 starting at stream_ordering 1222
2022-11-20 11:08:46,541 - synapse.federation.sender - 457 - INFO - process_event_queue_for_federation-15 - Unexpectedly did not have cached prev group for $eHVT50KcZqnWBw4gu1K4M7z4Tn6R0IWUGSjRHxu6gpk
2022-11-20 11:08:46,548 - synapse.push.httppusher - 225 - INFO - httppush.process-17 - Processing 0 unprocessed push actions for @yyyy:xxxx.com/im.vector.app.android/fgQCfmGEQBaZEM7NzVXJAl:APA91bGDP93cyMrollz6TZEx02CuYE_d1bUkK6uIyjAhcjOKy_DgxMgpvij60a27ExYzLQ3v2KASBgy8bC1Tx3s01TBNPPOtL6fI_lxk8Ow0gqB3Pbb4q8ZYmH2BNFP8KlM1kaP8mJ1A starting at stream_ordering 1226
2022-11-20 11:08:46,564 - synapse.access.http.8008 - 460 - INFO - GET-997 - 10.0.0.1 - 8008 - {@ssss:xxxx.com} Processed request: 1.785sec/0.004sec (0.013sec, 0.001sec) (0.007sec/0.016sec/3) 3246B 200 "GET /_matrix/client/r0/sync?filter=0&set_presence=online&timeout=30000&since=s1226_8353_0_316_396_4_145_79_0 HTTP/1.1" "Element/1.5.7 (Xiaomi 21051182C; Android 13; TD1A.221105.003; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [0 dbevts]
2022-11-20 11:08:46,565 - synapse.access.http.8008 - 460 - INFO - GET-998 - 10.0.0.6 - 8008 - {@ssss:xxxx.com} Processed request: 1.779sec/0.004sec (0.009sec, 0.000sec) (0.007sec/0.014sec/3) 3246B 200 "GET /_matrix/client/r0/sync?filter=0&set_presence=offline&timeout=6000&since=s1226_8353_0_316_396_4_145_79_0 HTTP/1.1" "Element/1.5.4 (Xiaomi 2112123AC; Android 12; SKQ1.211230.001 test-keys; Flavour FDroid; MatrixAndroidSdk2 1.5.4)" [1 dbevts]
2022-11-20 11:08:46,566 - synapse.access.http.8008 - 460 - INFO - GET-996 - 172.58.249.6 - 8008 - {@yyyy:xxxx.com} Processed request: 1.832sec/0.002sec (0.009sec, 0.002sec) (0.015sec/0.030sec/4) 3309B 200 "GET /_matrix/client/r0/sync?filter=1&set_presence=online&timeout=30000&since=s1226_8353_0_316_396_4_145_79_0 HTTP/1.1" "Element/1.5.7 (motorola XT2125-4; Android 12; S1RN32.55-16-2; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [0 dbevts]
2022-11-20 11:08:46,774 - synapse.rest.media.v1.media_repository - 476 - WARNING - GET-1009 - Not retrying destination 'chat.xxxx.com'
2022-11-20 11:08:46,775 - synapse.http.server - 108 - INFO - GET-1009 - <XForwardedForRequest at 0x7fddb8ae7ee0 method='GET' uri='/_matrix/media/r0/thumbnail/chat.xxxx.com/cAZsTMsbsIbaZBzZoCVefESp?width=250&height=250&method=scale' clientproto='HTTP/1.1' site='8008'> SynapseError: 502 - Failed to fetch remote media
2022-11-20 11:08:46,777 - synapse.access.http.8008 - 460 - INFO - GET-1009 - 172.58.249.6 - 8008 - {None} Processed request: 0.005sec/0.001sec (0.001sec, 0.000sec) (0.001sec/0.001sec/1) 62B 502 "GET /_matrix/media/r0/thumbnail/chat.xxxx.com/cAZsTMsbsIbaZBzZoCVefESp?width=250&height=250&method=scale HTTP/1.1" "Element/1.5.7 (motorola XT2125-4; Android 12; S1RN32.55-16-2; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [0 dbevts]
2022-11-20 11:08:47,045 - synapse.rest.media.v1.media_repository - 476 - WARNING - GET-1014 - Not retrying destination 'chat.xxxx.com'
2022-11-20 11:08:47,046 - synapse.http.server - 108 - INFO - GET-1014 - <XForwardedForRequest at 0x7fddb6e9f310 method='GET' uri='/_matrix/media/r0/thumbnail/chat.xxxx.com/cAZsTMsbsIbaZBzZoCVefESp?width=250&height=250&method=scale' clientproto='HTTP/1.1' site='8008'> SynapseError: 502 - Failed to fetch remote media
2022-11-20 11:08:47,048 - synapse.access.http.8008 - 460 - INFO - GET-1014 - 172.58.249.6 - 8008 - {None} Processed request: 0.006sec/0.001sec (0.001sec, 0.000sec) (0.001sec/0.001sec/1) 62B 502 "GET /_matrix/media/r0/thumbnail/chat.xxxx.com/cAZsTMsbsIbaZBzZoCVefESp?width=250&height=250&method=scale HTTP/1.1" "Element/1.5.7 (motorola XT2125-4; Android 12; S1RN32.55-16-2; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [0 dbevts]
2022-11-20 11:08:47,417 - synapse.http.client - 460 - INFO - httppush.process-16 - Received response to POST https://matrix.org/_matrix/push/v1/notify: 200
2022-11-20 11:08:47,689 - synapse.access.http.8008 - 460 - INFO - PUT-1011 - 10.0.0.161 - 8008 - {@ssss:xxxx.com} Processed request: 0.844sec/0.004sec (0.009sec, 0.000sec) (0.003sec/0.606sec/5) 59B 200 "PUT /_matrix/client/r0/rooms/!MHURLalqKLvNdeLuoC:xxxx.com/send/m.room.encrypted/$local.bc074190-00e1-4670-b8d3-7ece3c68757b HTTP/1.1" "Element/1.5.7 (Xiaomi 21051182C; Android 13; TD1A.221105.003; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [0 dbevts]
2022-11-20 11:08:47,715 - synapse.push.httppusher - 225 - INFO - httppush.process-18 - Processing 0 unprocessed push actions for @ssss:xxxx.com/im.vector.app.android/c5ExiLQzRU2ZU1TUmr-Ztb:APA91bEvl0jRCcDMoyNlEhcEIzV671hwNf5FNd5VTyutaGibNylPorDNloTwQ0V9aNN3mMgEZRdoAaKWLazHxtiaazGYgFnatojmFwXtBBiBa4WyxG0wSMEDgaXGjQU7geXx5dmjPWF5 starting at stream_ordering 1227
2022-11-20 11:08:47,719 - synapse.federation.sender - 457 - INFO - process_event_queue_for_federation-16 - Unexpectedly did not have cached prev group for $PzDXd7eSlCskHmIQE4-t4sIj7mfeSHKQf6FuiaH6R9s
2022-11-20 11:08:47,722 - synapse.push.httppusher - 225 - INFO - httppush.process-19 - Processing 1 unprocessed push actions for @yyyy:xxxx.com/im.vector.app.android/fgQCfmGEQBaZEM7NzVXJAl:APA91bGDP93cyMrollz6TZEx02CuYE_d1bUkK6uIyjAhcjOKy_DgxMgpvij60a27ExYzLQ3v2KASBgy8bC1Tx3s01TBNPPOtL6fI_lxk8Ow0gqB3Pbb4q8ZYmH2BNFP8KlM1kaP8mJ1A starting at stream_ordering 1226
2022-11-20 11:08:47,726 - synapse.access.http.8008 - 460 - INFO - GET-1010 - 10.0.0.1 - 8008 - {@ssss:xxxx.com} Processed request: 0.944sec/0.000sec (0.004sec, 0.001sec) (0.007sec/0.008sec/3) 1389B 200 "GET /_matrix/client/r0/sync?filter=0&set_presence=online&timeout=30000&since=s1227_8353_0_316_396_4_145_79_0 HTTP/1.1" "Element/1.5.7 (Xiaomi 21051182C; Android 13; TD1A.221105.003; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [1 dbevts]
2022-11-20 11:08:47,727 - synapse.access.http.8008 - 460 - INFO - GET-1008 - 10.0.0.6 - 8008 - {@ssss:xxxx.com} Processed request: 0.979sec/0.001sec (0.003sec, 0.001sec) (0.009sec/0.018sec/4) 1326B 200 "GET /_matrix/client/r0/sync?filter=0&set_presence=online&timeout=30000&since=s1227_8353_0_316_396_4_145_79_0 HTTP/1.1" "Element/1.5.4 (Xiaomi 2112123AC; Android 12; SKQ1.211230.001 test-keys; Flavour FDroid; MatrixAndroidSdk2 1.5.4)" [0 dbevts]
2022-11-20 11:08:47,728 - synapse.access.http.8008 - 460 - INFO - GET-1013 - 172.58.249.6 - 8008 - {@yyyy:xxxx.com} Processed request: 0.837sec/0.001sec (0.006sec, 0.000sec) (0.011sec/0.013sec/4) 1326B 200 "GET /_matrix/client/r0/sync?filter=1&set_presence=online&timeout=30000&since=s1227_8353_0_316_396_4_145_79_0 HTTP/1.1" "Element/1.5.7 (motorola XT2125-4; Android 12; S1RN32.55-16-2; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [0 dbevts]
2022-11-20 11:08:47,834 - synapse.access.http.8008 - 460 - INFO - PUT-1012 - 172.58.249.6 - 8008 - {@yyyy:xxxx.com} Processed request: 0.969sec/0.002sec (0.008sec, 0.000sec) (0.002sec/0.692sec/4) 59B 200 "PUT /_matrix/client/r0/rooms/!MHURLalqKLvNdeLuoC:xxxx.com/send/m.room.encrypted/$local.993480e4-8ff7-46fa-924e-8a797802a7b1 HTTP/1.1" "Element/1.5.7 (motorola XT2125-4; Android 12; S1RN32.55-16-2; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [0 dbevts]
2022-11-20 11:08:47,838 - synapse.federation.sender - 457 - INFO - process_event_queue_for_federation-17 - Unexpectedly did not have cached prev group for $2G3_VT843PFYfulTZZufEwak2KAM6XOgrPBNRPYJS10
2022-11-20 11:08:47,840 - synapse.push.httppusher - 225 - INFO - httppush.process-20 - Processing 1 unprocessed push actions for @ssss:xxxx.com/im.vector.app.android/c5ExiLQzRU2ZU1TUmr-Ztb:APA91bEvl0jRCcDMoyNlEhcEIzV671hwNf5FNd5VTyutaGibNylPorDNloTwQ0V9aNN3mMgEZRdoAaKWLazHxtiaazGYgFnatojmFwXtBBiBa4WyxG0wSMEDgaXGjQU7geXx5dmjPWF5 starting at stream_ordering 1227
2022-11-20 11:08:47,957 - synapse.rest.media.v1.media_repository - 476 - WARNING - GET-1016 - Not retrying destination 'chat.xxxx.com'
2022-11-20 11:08:47,958 - synapse.http.server - 108 - INFO - GET-1016 - <XForwardedForRequest at 0x7fddb6ec8a30 method='GET' uri='/_matrix/media/r0/thumbnail/chat.xxxx.com/cAZsTMsbsIbaZBzZoCVefESp?width=250&height=250&method=scale' clientproto='HTTP/1.1' site='8008'> SynapseError: 502 - Failed to fetch remote media
2022-11-20 11:08:47,966 - synapse.access.http.8008 - 460 - INFO - GET-1016 - 172.58.249.6 - 8008 - {None} Processed request: 0.015sec/0.008sec (0.003sec, 0.000sec) (0.005sec/0.002sec/1) 62B 502 "GET /_matrix/media/r0/thumbnail/chat.xxxx.com/cAZsTMsbsIbaZBzZoCVefESp?width=250&height=250&method=scale HTTP/1.1" "Element/1.5.7 (motorola XT2125-4; Android 12; S1RN32.55-16-2; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [0 dbevts]
2022-11-20 11:08:47,967 - synapse.access.http.8008 - 460 - INFO - GET-1015 - 10.0.0.1 - 8008 - {@ssss:xxxx.com} Processed request: 0.025sec/0.004sec (0.007sec, 0.000sec) (0.002sec/0.010sec/2) 4420B 200 "GET /_matrix/client/r0/sync?filter=0&set_presence=online&timeout=30000&since=s1228_8353_0_316_396_4_145_79_0 HTTP/1.1" "Element/1.5.7 (Xiaomi 21051182C; Android 13; TD1A.221105.003; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [0 dbevts]
2022-11-20 11:08:47,967 - synapse.access.http.8008 - 460 - INFO - GET-1017 - 10.0.0.6 - 8008 - {@ssss:xxxx.com} Processed request: 0.021sec/0.002sec (0.005sec, 0.000sec) (0.003sec/0.006sec/2) 4420B 200 "GET /_matrix/client/r0/sync?filter=0&set_presence=online&timeout=30000&since=s1228_8353_0_316_396_4_145_79_0 HTTP/1.1" "Element/1.5.4 (Xiaomi 2112123AC; Android 12; SKQ1.211230.001 test-keys; Flavour FDroid; MatrixAndroidSdk2 1.5.4)" [0 dbevts]
2022-11-20 11:08:48,029 - synapse.http.client - 460 - INFO - httppush.process-19 - Received response to POST https://matrix.org/_matrix/push/v1/notify: 200
2022-11-20 11:08:48,068 - synapse.push.httppusher - 225 - INFO - httppush.process-19 - Processing 0 unprocessed push actions for @yyyy:xxxx.com/im.vector.app.android/fgQCfmGEQBaZEM7NzVXJAl:APA91bGDP93cyMrollz6TZEx02CuYE_d1bUkK6uIyjAhcjOKy_DgxMgpvij60a27ExYzLQ3v2KASBgy8bC1Tx3s01TBNPPOtL6fI_lxk8Ow0gqB3Pbb4q8ZYmH2BNFP8KlM1kaP8mJ1A starting at stream_ordering 1228
2022-11-20 11:08:48,072 - synapse.access.http.8008 - 460 - INFO - GET-1019 - 172.58.249.6 - 8008 - {@yyyy:xxxx.com} Processed request: 0.007sec/0.000sec (0.002sec, 0.000sec) (0.001sec/0.003sec/3) 4483B 200 "GET /_matrix/client/r0/sync?filter=1&set_presence=online&timeout=30000&since=s1228_8353_0_316_396_4_145_79_0 HTTP/1.1" "Element/1.5.7 (motorola XT2125-4; Android 12; S1RN32.55-16-2; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [0 dbevts]
2022-11-20 11:08:48,098 - synapse.http.client - 460 - INFO - httppush.process-20 - Received response to POST https://matrix.org/_matrix/push/v1/notify: 200
2022-11-20 11:08:48,206 - synapse.access.http.8008 - 460 - INFO - PUT-1018 - 172.58.249.6 - 8008 - {@yyyy:xxxx.com} Processed request: 0.164sec/0.001sec (0.008sec, 0.001sec) (0.004sec/0.034sec/5) 59B 200 "PUT /_matrix/client/r0/rooms/!MHURLalqKLvNdeLuoC:xxxx.com/send/m.room.encrypted/$local.ab9fb33f-8f0f-419f-891e-b0027073bd5e HTTP/1.1" "Element/1.5.7 (motorola XT2125-4; Android 12; S1RN32.55-16-2; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [0 dbevts]
2022-11-20 11:08:48,217 - synapse.federation.sender - 457 - INFO - process_event_queue_for_federation-18 - Unexpectedly did not have cached prev group for $08fzrImGVajYCzxkhLY0XYnA7COx12xZn8XSGaWmMu4
2022-11-20 11:08:48,219 - synapse.push.httppusher - 225 - INFO - httppush.process-21 - Processing 1 unprocessed push actions for @ssss:xxxx.com/im.vector.app.android/c5ExiLQzRU2ZU1TUmr-Ztb:APA91bEvl0jRCcDMoyNlEhcEIzV671hwNf5FNd5VTyutaGibNylPorDNloTwQ0V9aNN3mMgEZRdoAaKWLazHxtiaazGYgFnatojmFwXtBBiBa4WyxG0wSMEDgaXGjQU7geXx5dmjPWF5 starting at stream_ordering 1229
2022-11-20 11:08:48,232 - synapse.access.http.8008 - 460 - INFO - GET-1020 - 10.0.0.6 - 8008 - {@ssss:xxxx.com} Processed request: 0.117sec/0.005sec (0.006sec, 0.001sec) (0.007sec/0.010sec/4) 3374B 200 "GET /_matrix/client/r0/sync?filter=0&set_presence=online&timeout=30000&since=s1229_8353_0_316_396_4_145_79_0 HTTP/1.1" "Element/1.5.4 (Xiaomi 2112123AC; Android 12; SKQ1.211230.001 test-keys; Flavour FDroid; MatrixAndroidSdk2 1.5.4)" [0 dbevts]
2022-11-20 11:08:48,233 - synapse.access.http.8008 - 460 - INFO - GET-1021 - 10.0.0.1 - 8008 - {@ssss:xxxx.com} Processed request: 0.091sec/0.004sec (0.004sec, 0.001sec) (0.005sec/0.003sec/3) 3374B 200 "GET /_matrix/client/r0/sync?filter=0&set_presence=online&timeout=30000&since=s1229_8353_0_316_396_4_145_79_0 HTTP/1.1" "Element/1.5.7 (Xiaomi 21051182C; Android 13; TD1A.221105.003; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [1 dbevts]
2022-11-20 11:08:48,281 - synapse.rest.media.v1.media_repository - 476 - WARNING - GET-1023 - Not retrying destination 'chat.xxxx.com'
2022-11-20 11:08:48,281 - synapse.http.server - 108 - INFO - GET-1023 - <XForwardedForRequest at 0x7fddb6eac5e0 method='GET' uri='/_matrix/media/r0/thumbnail/chat.xxxx.com/cAZsTMsbsIbaZBzZoCVefESp?width=250&height=250&method=scale' clientproto='HTTP/1.1' site='8008'> SynapseError: 502 - Failed to fetch remote media
2022-11-20 11:08:48,282 - synapse.access.http.8008 - 460 - INFO - GET-1023 - 172.58.249.6 - 8008 - {None} Processed request: 0.001sec/0.000sec (0.000sec, 0.000sec) (0.000sec/0.000sec/1) 62B 502 "GET /_matrix/media/r0/thumbnail/chat.xxxx.com/cAZsTMsbsIbaZBzZoCVefESp?width=250&height=250&method=scale HTTP/1.1" "Element/1.5.7 (motorola XT2125-4; Android 12; S1RN32.55-16-2; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [0 dbevts]
2022-11-20 11:08:48,285 - synapse.access.http.8008 - 460 - INFO - GET-1022 - 172.58.249.6 - 8008 - {@yyyy:xxxx.com} Processed request: 0.005sec/0.000sec (0.001sec, 0.000sec) (0.001sec/0.002sec/3) 3437B 200 "GET /_matrix/client/r0/sync?filter=1&set_presence=online&timeout=30000&since=s1229_8353_0_316_396_4_145_79_0 HTTP/1.1" "Element/1.5.7 (motorola XT2125-4; Android 12; S1RN32.55-16-2; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [0 dbevts]
2022-11-20 11:08:48,431 - synapse.http.client - 460 - INFO - httppush.process-21 - Received response to POST https://matrix.org/_matrix/push/v1/notify: 200
2022-11-20 11:08:48,550 - synapse.rest.media.v1.media_repository - 476 - WARNING - GET-1028 - Not retrying destination 'chat.xxxx.com'
2022-11-20 11:08:48,550 - synapse.http.server - 108 - INFO - GET-1028 - <XForwardedForRequest at 0x7fddb7d586d0 method='GET' uri='/_matrix/media/r0/thumbnail/chat.xxxx.com/cAZsTMsbsIbaZBzZoCVefESp?width=250&height=250&method=scale' clientproto='HTTP/1.1' site='8008'> SynapseError: 502 - Failed to fetch remote media
2022-11-20 11:08:48,551 - synapse.access.http.8008 - 460 - INFO - GET-1028 - 172.58.249.6 - 8008 - {None} Processed request: 0.002sec/0.000sec (0.000sec, 0.000sec) (0.000sec/0.000sec/1) 62B 502 "GET /_matrix/media/r0/thumbnail/chat.xxxx.com/cAZsTMsbsIbaZBzZoCVefESp?width=250&height=250&method=scale HTTP/1.1" "Element/1.5.7 (motorola XT2125-4; Android 12; S1RN32.55-16-2; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [0 dbevts]
2022-11-20 11:08:48,693 - synapse.access.http.8008 - 460 - INFO - PUT-1026 - 10.0.0.1 - 8008 - {@ssss:xxxx.com} Processed request: 0.168sec/0.006sec (0.006sec, 0.002sec) (0.006sec/0.038sec/5) 59B 200 "PUT /_matrix/client/r0/rooms/!MHURLalqKLvNdeLuoC:xxxx.com/send/m.room.encrypted/$local.3a5e5436-5a7d-4e01-af3f-4ba2d556ab0b HTTP/1.1" "Element/1.5.7 (Xiaomi 21051182C; Android 13; TD1A.221105.003; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [0 dbevts]
2022-11-20 11:08:48,716 - synapse.federation.sender - 457 - INFO - process_event_queue_for_federation-19 - Unexpectedly did not have cached prev group for $XGcIZbP-nHovY1e2oc3qmZj64d47t1THu09KfckIyD4
2022-11-20 11:08:48,728 - synapse.push.httppusher - 225 - INFO - httppush.process-23 - Processing 1 unprocessed push actions for @yyyy:xxxx.com/im.vector.app.android/fgQCfmGEQBaZEM7NzVXJAl:APA91bGDP93cyMrollz6TZEx02CuYE_d1bUkK6uIyjAhcjOKy_DgxMgpvij60a27ExYzLQ3v2KASBgy8bC1Tx3s01TBNPPOtL6fI_lxk8Ow0gqB3Pbb4q8ZYmH2BNFP8KlM1kaP8mJ1A starting at stream_ordering 1228
2022-11-20 11:08:48,729 - synapse.push.httppusher - 225 - INFO - httppush.process-22 - Processing 0 unprocessed push actions for @ssss:xxxx.com/im.vector.app.android/c5ExiLQzRU2ZU1TUmr-Ztb:APA91bEvl0jRCcDMoyNlEhcEIzV671hwNf5FNd5VTyutaGibNylPorDNloTwQ0V9aNN3mMgEZRdoAaKWLazHxtiaazGYgFnatojmFwXtBBiBa4WyxG0wSMEDgaXGjQU7geXx5dmjPWF5 starting at stream_ordering 1230
2022-11-20 11:08:48,742 - synapse.access.http.8008 - 460 - INFO - GET-1025 - 172.58.249.6 - 8008 - {@yyyy:xxxx.com} Processed request: 0.237sec/0.005sec (0.011sec, 0.002sec) (0.008sec/0.019sec/4) 3246B 200 "GET /_matrix/client/r0/sync?filter=1&set_presence=online&timeout=30000&since=s1230_8353_0_316_396_4_145_79_0 HTTP/1.1" "Element/1.5.7 (motorola XT2125-4; Android 12; S1RN32.55-16-2; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [1 dbevts]
2022-11-20 11:08:48,753 - synapse.access.http.8008 - 460 - INFO - GET-1024 - 10.0.0.6 - 8008 - {@ssss:xxxx.com} Processed request: 0.301sec/0.003sec (0.011sec, 0.001sec) (0.021sec/0.028sec/4) 3246B 200 "GET /_matrix/client/r0/sync?filter=0&set_presence=online&timeout=30000&since=s1230_8353_0_316_396_4_145_79_0 HTTP/1.1" "Element/1.5.4 (Xiaomi 2112123AC; Android 12; SKQ1.211230.001 test-keys; Flavour FDroid; MatrixAndroidSdk2 1.5.4)" [0 dbevts]
2022-11-20 11:08:48,754 - synapse.access.http.8008 - 460 - INFO - GET-1027 - 10.0.0.161 - 8008 - {@ssss:xxxx.com} Processed request: 0.208sec/0.003sec (0.010sec, 0.000sec) (0.007sec/0.018sec/3) 3309B 200 "GET /_matrix/client/r0/sync?filter=0&set_presence=online&timeout=30000&since=s1230_8353_0_316_396_4_145_79_0 HTTP/1.1" "Element/1.5.7 (Xiaomi 21051182C; Android 13; TD1A.221105.003; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [0 dbevts]
2022-11-20 11:08:48,903 - synapse.http.client - 460 - INFO - httppush.process-23 - Received response to POST https://matrix.org/_matrix/push/v1/notify: 200
2022-11-20 11:08:49,084 - synapse.rest.media.v1.media_repository - 476 - WARNING - GET-1032 - Not retrying destination 'chat.xxxx.com'
2022-11-20 11:08:49,085 - synapse.http.server - 108 - INFO - GET-1032 - <XForwardedForRequest at 0x7fddb53d5130 method='GET' uri='/_matrix/media/r0/thumbnail/chat.xxxx.com/cAZsTMsbsIbaZBzZoCVefESp?width=250&height=250&method=scale' clientproto='HTTP/1.1' site='8008'> SynapseError: 502 - Failed to fetch remote media
2022-11-20 11:08:49,086 - synapse.access.http.8008 - 460 - INFO - GET-1032 - 172.58.249.6 - 8008 - {None} Processed request: 0.005sec/0.001sec (0.001sec, 0.000sec) (0.001sec/0.001sec/1) 62B 502 "GET /_matrix/media/r0/thumbnail/chat.xxxx.com/cAZsTMsbsIbaZBzZoCVefESp?width=250&height=250&method=scale HTTP/1.1" "Element/1.5.7 (motorola XT2125-4; Android 12; S1RN32.55-16-2; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [0 dbevts]
2022-11-20 11:08:49,125 - synapse.access.http.8008 - 460 - INFO - GET-1033 - 172.58.249.6 - 8008 - {@yyyy:xxxx.com} Processed request: 0.006sec/0.000sec (0.002sec, 0.000sec) (0.000sec/0.002sec/3) 2329B 200 "GET /_matrix/client/r0/sync?filter=1&set_presence=online&timeout=30000&since=s1231_8353_0_316_396_4_145_79_0 HTTP/1.1" "Element/1.5.7 (motorola XT2125-4; Android 12; S1RN32.55-16-2; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [1 dbevts]
2022-11-20 11:08:49,136 - synapse.access.http.8008 - 460 - INFO - PUT-1029 - 10.0.0.1 - 8008 - {@ssss:xxxx.com} Processed request: 0.247sec/0.001sec (0.007sec, 0.002sec) (0.003sec/0.046sec/5) 59B 200 "PUT /_matrix/client/r0/rooms/!MHURLalqKLvNdeLuoC:xxxx.com/send/m.room.encrypted/$local.73f099a2-8d88-4955-b904-ec6d75d460b7 HTTP/1.1" "Element/1.5.7 (Xiaomi 21051182C; Android 13; TD1A.221105.003; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [0 dbevts]
2022-11-20 11:08:49,140 - synapse.federation.sender - 457 - INFO - process_event_queue_for_federation-20 - Unexpectedly did not have cached prev group for $wDHPg8MZPe79cH6iY5lHRk9xDBfR_xb5IUVwlBmtNr0
2022-11-20 11:08:49,141 - synapse.push.httppusher - 225 - INFO - httppush.process-24 - Processing 1 unprocessed push actions for @yyyy:xxxx.com/im.vector.app.android/fgQCfmGEQBaZEM7NzVXJAl:APA91bGDP93cyMrollz6TZEx02CuYE_d1bUkK6uIyjAhcjOKy_DgxMgpvij60a27ExYzLQ3v2KASBgy8bC1Tx3s01TBNPPOtL6fI_lxk8Ow0gqB3Pbb4q8ZYmH2BNFP8KlM1kaP8mJ1A starting at stream_ordering 1231
2022-11-20 11:08:49,143 - synapse.access.http.8008 - 460 - INFO - GET-1031 - 10.0.0.6 - 8008 - {@ssss:xxxx.com} Processed request: 0.108sec/0.001sec (0.004sec, 0.000sec) (0.003sec/0.006sec/4) 2329B 200 "GET /_matrix/client/r0/sync?filter=0&set_presence=online&timeout=30000&since=s1231_8353_0_316_396_4_145_79_0 HTTP/1.1" "Element/1.5.4 (Xiaomi 2112123AC; Android 12; SKQ1.211230.001 test-keys; Flavour FDroid; MatrixAndroidSdk2 1.5.4)" [0 dbevts]
2022-11-20 11:08:49,144 - synapse.access.http.8008 - 460 - INFO - GET-1030 - 10.0.0.161 - 8008 - {@ssss:xxxx.com} Processed request: 0.146sec/0.000sec (0.002sec, 0.000sec) (0.001sec/0.004sec/3) 2392B 200 "GET /_matrix/client/r0/sync?filter=0&set_presence=online&timeout=30000&since=s1231_8353_0_316_396_4_145_79_0 HTTP/1.1" "Element/1.5.7 (Xiaomi 21051182C; Android 13; TD1A.221105.003; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [0 dbevts]
2022-11-20 11:08:49,310 - synapse.http.client - 460 - INFO - httppush.process-24 - Received response to POST https://matrix.org/_matrix/push/v1/notify: 200
2022-11-20 11:08:49,426 - synapse.rest.media.v1.media_repository - 476 - WARNING - GET-1037 - Not retrying destination 'chat.xxxx.com'
2022-11-20 11:08:49,427 - synapse.http.server - 108 - INFO - GET-1037 - <XForwardedForRequest at 0x7fddb60f4f10 method='GET' uri='/_matrix/media/r0/thumbnail/chat.xxxx.com/cAZsTMsbsIbaZBzZoCVefESp?width=250&height=250&method=scale' clientproto='HTTP/1.1' site='8008'> SynapseError: 502 - Failed to fetch remote media
2022-11-20 11:08:49,429 - synapse.access.http.8008 - 460 - INFO - GET-1037 - 172.58.249.6 - 8008 - {None} Processed request: 0.007sec/0.001sec (0.002sec, 0.001sec) (0.001sec/0.002sec/1) 62B 502 "GET /_matrix/media/r0/thumbnail/chat.xxxx.com/cAZsTMsbsIbaZBzZoCVefESp?width=250&height=250&method=scale HTTP/1.1" "Element/1.5.7 (motorola XT2125-4; Android 12; S1RN32.55-16-2; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [0 dbevts]
2022-11-20 11:08:49,444 - synapse.http.client - 460 - INFO - http_pusher.on_new_receipts-3 - Received response to POST https://matrix.org/_matrix/push/v1/notify: 200
2022-11-20 11:08:57,751 - synapse.storage.databases.main.event_push_actions - 1305 - INFO - rotate_notifs-62 - Rotating notifications
2022-11-20 11:08:57,754 - synapse.storage.databases.main.event_push_actions - 1528 - INFO - rotate_notifs-62 - Rotating notifications up to: 1232
2022-11-20 11:08:57,758 - synapse.storage.databases.main.event_push_actions - 1627 - INFO - rotate_notifs-62 - Rotating notifications, handling 2 rows
2022-11-20 11:08:57,813 - synapse.storage.databases.main.event_push_actions - 1720 - INFO - rotate_notifs-62 - Rotating notifications, deleted 0 push actions
2022-11-20 11:08:58,039 - synapse.util.caches.lrucache - 212 - INFO - LruCache._expire_old_entries-62 - Dropped 0 items from caches
2022-11-20 11:09:01,365 - synapse.rest.media.v1.media_repository - 476 - WARNING - GET-1039 - Not retrying destination 'chat.xxxx.com'
2022-11-20 11:09:01,366 - synapse.http.server - 108 - INFO - GET-1039 - <XForwardedForRequest at 0x7fddb7cfc4c0 method='GET' uri='/_matrix/media/r0/thumbnail/chat.xxxx.com/cAZsTMsbsIbaZBzZoCVefESp?width=250&height=250&method=scale' clientproto='HTTP/1.1' site='8008'> SynapseError: 502 - Failed to fetch remote media
2022-11-20 11:09:01,368 - synapse.access.http.8008 - 460 - INFO - GET-1039 - 172.58.249.6 - 8008 - {None} Processed request: 0.006sec/0.001sec (0.000sec, 0.001sec) (0.001sec/0.001sec/1) 62B 502 "GET /_matrix/media/r0/thumbnail/chat.xxxx.com/cAZsTMsbsIbaZBzZoCVefESp?width=250&height=250&method=scale HTTP/1.1" "Element/1.5.7 (motorola XT2125-4; Android 12; S1RN32.55-16-2; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [0 dbevts]
2022-11-20 11:09:01,478 - synapse.rest.media.v1.media_repository - 476 - WARNING - GET-1041 - Not retrying destination 'chat.xxxx.com'
2022-11-20 11:09:01,479 - synapse.http.server - 108 - INFO - GET-1041 - <XForwardedForRequest at 0x7fddb607dbb0 method='GET' uri='/_matrix/media/r0/thumbnail/chat.xxxx.com/cAZsTMsbsIbaZBzZoCVefESp?width=250&height=250&method=scale' clientproto='HTTP/1.1' site='8008'> SynapseError: 502 - Failed to fetch remote media
2022-11-20 11:09:01,486 - synapse.access.http.8008 - 460 - INFO - GET-1041 - 172.58.249.6 - 8008 - {None} Processed request: 0.008sec/0.007sec (0.004sec, 0.001sec) (0.001sec/0.002sec/1) 62B 502 "GET /_matrix/media/r0/thumbnail/chat.xxxx.com/cAZsTMsbsIbaZBzZoCVefESp?width=250&height=250&method=scale HTTP/1.1" "Element/1.5.7 (motorola XT2125-4; Android 12; S1RN32.55-16-2; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [0 dbevts]
2022-11-20 11:09:01,598 - synapse.access.http.8008 - 460 - INFO - PUT-1040 - 172.58.249.6 - 8008 - {@yyyy:xxxx.com} Processed request: 0.132sec/0.006sec (0.006sec, 0.002sec) (0.003sec/0.036sec/5) 59B 200 "PUT /_matrix/client/r0/rooms/!MHURLalqKLvNdeLuoC:xxxx.com/send/m.room.encrypted/$local.096bd076-dbd5-4516-bc25-ced40ae0c5bc HTTP/1.1" "Element/1.5.7 (motorola XT2125-4; Android 12; S1RN32.55-16-2; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [0 dbevts]
2022-11-20 11:09:01,627 - synapse.federation.sender - 457 - INFO - process_event_queue_for_federation-21 - Unexpectedly did not have cached prev group for $U_op9uxJP7oFWOyLsOg7MPNY9sXE9cPjSQ67GQNmbZY
2022-11-20 11:09:01,635 - synapse.push.httppusher - 225 - INFO - httppush.process-25 - Processing 1 unprocessed push actions for @ssss:xxxx.com/im.vector.app.android/c5ExiLQzRU2ZU1TUmr-Ztb:APA91bEvl0jRCcDMoyNlEhcEIzV671hwNf5FNd5VTyutaGibNylPorDNloTwQ0V9aNN3mMgEZRdoAaKWLazHxtiaazGYgFnatojmFwXtBBiBa4WyxG0wSMEDgaXGjQU7geXx5dmjPWF5 starting at stream_ordering 1230
2022-11-20 11:09:01,635 - synapse.push.httppusher - 225 - INFO - httppush.process-26 - Processing 0 unprocessed push actions for @yyyy:xxxx.com/im.vector.app.android/fgQCfmGEQBaZEM7NzVXJAl:APA91bGDP93cyMrollz6TZEx02CuYE_d1bUkK6uIyjAhcjOKy_DgxMgpvij60a27ExYzLQ3v2KASBgy8bC1Tx3s01TBNPPOtL6fI_lxk8Ow0gqB3Pbb4q8ZYmH2BNFP8KlM1kaP8mJ1A starting at stream_ordering 1232
2022-11-20 11:09:01,644 - synapse.access.http.8008 - 460 - INFO - GET-1035 - 10.0.0.1 - 8008 - {@ssss:xxxx.com} Processed request: 12.320sec/0.002sec (0.005sec, 0.001sec) (0.008sec/0.008sec/3) 1305B 200 "GET /_matrix/client/r0/sync?filter=0&set_presence=online&timeout=30000&since=s1232_8353_0_316_396_4_145_79_0 HTTP/1.1" "Element/1.5.7 (Xiaomi 21051182C; Android 13; TD1A.221105.003; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [1 dbevts]
2022-11-20 11:09:01,645 - synapse.access.http.8008 - 460 - INFO - GET-1036 - 172.58.249.6 - 8008 - {@yyyy:xxxx.com} Processed request: 12.226sec/0.004sec (0.011sec, 0.002sec) (0.011sec/0.025sec/4) 1368B 200 "GET /_matrix/client/r0/sync?filter=1&set_presence=online&timeout=30000&since=s1232_8353_0_316_396_4_145_79_0 HTTP/1.1" "Element/1.5.7 (motorola XT2125-4; Android 12; S1RN32.55-16-2; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [0 dbevts]
2022-11-20 11:09:01,645 - synapse.access.http.8008 - 460 - INFO - GET-1034 - 10.0.0.6 - 8008 - {@ssss:xxxx.com} Processed request: 12.327sec/0.003sec (0.009sec, 0.002sec) (0.015sec/0.020sec/4) 1305B 200 "GET /_matrix/client/r0/sync?filter=0&set_presence=online&timeout=30000&since=s1232_8353_0_316_396_4_145_79_0 HTTP/1.1" "Element/1.5.4 (Xiaomi 2112123AC; Android 12; SKQ1.211230.001 test-keys; Flavour FDroid; MatrixAndroidSdk2 1.5.4)" [0 dbevts]
2022-11-20 11:09:01,755 - synapse.rest.media.v1.media_repository - 476 - WARNING - GET-1042 - Not retrying destination 'chat.xxxx.com'
2022-11-20 11:09:01,756 - synapse.http.server - 108 - INFO - GET-1042 - <XForwardedForRequest at 0x7fddb6e59ca0 method='GET' uri='/_matrix/media/r0/thumbnail/chat.xxxx.com/cAZsTMsbsIbaZBzZoCVefESp?width=250&height=250&method=scale' clientproto='HTTP/1.1' site='8008'> SynapseError: 502 - Failed to fetch remote media
2022-11-20 11:09:01,758 - synapse.access.http.8008 - 460 - INFO - GET-1042 - 172.58.249.6 - 8008 - {None} Processed request: 0.006sec/0.001sec (0.001sec, 0.000sec) (0.001sec/0.001sec/1) 62B 502 "GET /_matrix/media/r0/thumbnail/chat.xxxx.com/cAZsTMsbsIbaZBzZoCVefESp?width=250&height=250&method=scale HTTP/1.1" "Element/1.5.7 (motorola XT2125-4; Android 12; S1RN32.55-16-2; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [0 dbevts]
2022-11-20 11:09:01,815 - synapse.http.client - 460 - INFO - httppush.process-25 - Received response to POST https://matrix.org/_matrix/push/v1/notify: 200
2022-11-20 11:09:02,064 - synapse.rest.media.v1.media_repository - 476 - WARNING - GET-1046 - Not retrying destination 'chat.xxxx.com'
2022-11-20 11:09:02,065 - synapse.http.server - 108 - INFO - GET-1046 - <XForwardedForRequest at 0x7fddb6140400 method='GET' uri='/_matrix/media/r0/thumbnail/chat.xxxx.com/cAZsTMsbsIbaZBzZoCVefESp?width=250&height=250&method=scale' clientproto='HTTP/1.1' site='8008'> SynapseError: 502 - Failed to fetch remote media
2022-11-20 11:09:02,067 - synapse.access.http.8008 - 460 - INFO - GET-1046 - 172.58.249.6 - 8008 - {None} Processed request: 0.005sec/0.001sec (0.001sec, 0.000sec) (0.000sec/0.001sec/1) 62B 502 "GET /_matrix/media/r0/thumbnail/chat.xxxx.com/cAZsTMsbsIbaZBzZoCVefESp?width=250&height=250&method=scale HTTP/1.1" "Element/1.5.7 (motorola XT2125-4; Android 12; S1RN32.55-16-2; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [0 dbevts]
2022-11-20 11:09:02,374 - synapse.rest.media.v1.media_repository - 476 - WARNING - GET-1047 - Not retrying destination 'chat.xxxx.com'
2022-11-20 11:09:02,375 - synapse.http.server - 108 - INFO - GET-1047 - <XForwardedForRequest at 0x7fddb6e47ca0 method='GET' uri='/_matrix/media/r0/thumbnail/chat.xxxx.com/cAZsTMsbsIbaZBzZoCVefESp?width=250&height=250&method=scale' clientproto='HTTP/1.1' site='8008'> SynapseError: 502 - Failed to fetch remote media
2022-11-20 11:09:02,376 - synapse.access.http.8008 - 460 - INFO - GET-1047 - 172.58.249.6 - 8008 - {None} Processed request: 0.005sec/0.001sec (0.001sec, 0.000sec) (0.001sec/0.001sec/1) 62B 502 "GET /_matrix/media/r0/thumbnail/chat.xxxx.com/cAZsTMsbsIbaZBzZoCVefESp?width=250&height=250&method=scale HTTP/1.1" "Element/1.5.7 (motorola XT2125-4; Android 12; S1RN32.55-16-2; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [0 dbevts]
2022-11-20 11:09:03,019 - synapse.access.http.8008 - 460 - INFO - POST-1048 - 10.0.0.161 - 8008 - {@ssss:xxxx.com} Processed request: 0.108sec/0.004sec (0.005sec, 0.000sec) (0.004sec/0.091sec/5) 2B 200 "POST /_matrix/client/r0/rooms/!MHURLalqKLvNdeLuoC:xxxx.com/receipt/m.read/$U_op9uxJP7oFWOyLsOg7MPNY9sXE9cPjSQ67GQNmbZY HTTP/1.1" "Element/1.5.7 (Xiaomi 21051182C; Android 13; TD1A.221105.003; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [0 dbevts]
2022-11-20 11:09:03,042 - synapse.access.http.8008 - 460 - INFO - GET-1043 - 172.58.249.6 - 8008 - {@yyyy:xxxx.com} Processed request: 1.228sec/0.003sec (0.008sec, 0.002sec) (0.008sec/0.013sec/4) 680B 200 "GET /_matrix/client/r0/sync?filter=1&set_presence=online&timeout=30000&since=s1233_8353_0_316_396_4_145_79_0 HTTP/1.1" "Element/1.5.7 (motorola XT2125-4; Android 12; S1RN32.55-16-2; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [0 dbevts]
2022-11-20 11:09:03,043 - synapse.access.http.8008 - 460 - INFO - GET-1044 - 10.0.0.1 - 8008 - {@ssss:xxxx.com} Processed request: 1.171sec/0.008sec (0.006sec, 0.000sec) (0.004sec/0.006sec/2) 680B 200 "GET /_matrix/client/r0/sync?filter=0&set_presence=online&timeout=30000&since=s1233_8353_0_316_396_4_145_79_0 HTTP/1.1" "Element/1.5.7 (Xiaomi 21051182C; Android 13; TD1A.221105.003; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [0 dbevts]
2022-11-20 11:09:03,044 - synapse.access.http.8008 - 460 - INFO - GET-1045 - 10.0.0.6 - 8008 - {@ssss:xxxx.com} Processed request: 1.002sec/0.007sec (0.006sec, 0.000sec) (0.004sec/0.006sec/2) 680B 200 "GET /_matrix/client/r0/sync?filter=0&set_presence=online&timeout=30000&since=s1233_8353_0_316_396_4_145_79_0 HTTP/1.1" "Element/1.5.4 (Xiaomi 2112123AC; Android 12; SKQ1.211230.001 test-keys; Flavour FDroid; MatrixAndroidSdk2 1.5.4)" [0 dbevts]
2022-11-20 11:09:03,452 - synapse.rest.media.v1.media_repository - 476 - WARNING - GET-1051 - Not retrying destination 'chat.xxxx.com'
2022-11-20 11:09:03,452 - synapse.http.server - 108 - INFO - GET-1051 - <XForwardedForRequest at 0x7fddb6e68670 method='GET' uri='/_matrix/media/r0/thumbnail/chat.xxxx.com/cAZsTMsbsIbaZBzZoCVefESp?width=250&height=250&method=scale' clientproto='HTTP/1.1' site='8008'> SynapseError: 502 - Failed to fetch remote media
2022-11-20 11:09:03,453 - synapse.access.http.8008 - 460 - INFO - GET-1051 - 172.58.249.6 - 8008 - {None} Processed request: 0.002sec/0.000sec (0.000sec, 0.000sec) (0.000sec/0.000sec/1) 62B 502 "GET /_matrix/media/r0/thumbnail/chat.xxxx.com/cAZsTMsbsIbaZBzZoCVefESp?width=250&height=250&method=scale HTTP/1.1" "Element/1.5.7 (motorola XT2125-4; Android 12; S1RN32.55-16-2; Flavour GooglePlay; MatrixAndroidSdk2 1.5.7)" [0 dbevts]
2022-11-20 11:09:03,574 - synapse.http.client - 460 - INFO - http_pusher.on_new_receipts-4 - Received response to POST https://matrix.org/_matrix/push/v1/notify: 200
2022-11-20 11:09:09,516 - synapse.access.http.8008 - 460 - INFO - GET-974 - ::ffff:172.19.0.13 - 8008 - {@telegrambot:xxxx.com} Processed request: 30.003sec/0.001sec (0.003sec, 0.000sec) (0.001sec/0.002sec/1) 197B 200 "GET /_matrix/client/v3/sync?timeout=30000&since=s1223_8353_0_315_396_4_145_79_0&filter=0&set_presence=online HTTP/1.1" "mautrix-telegram/0.12.1+dev.5558fc71 mautrix-python/0.18.6 aiohttp/3.8.1 Python/3.10.5" [0 dbevts]
2022-11-20 11:09:12,085 - synapse.access.http.8008 - 460 - INFO - GET-978 - ::ffff:172.19.0.15 - 8008 - {@signalbot:xxxx.com} Processed request: 30.003sec/0.001sec (0.003sec, 0.000sec) (0.001sec/0.002sec/1) 197B 200 "GET /_matrix/client/v3/sync?timeout=30000&since=s1223_8353_0_315_396_4_145_79_0&filter=0&set_presence=online HTTP/1.1" "mautrix-signal/0.4.1+dev.3cbdac88 mautrix-python/0.18.6 aiohttp/3.8.1 Python/3.10.5" [0 dbevts]
J4gQBqqR commented 1 year ago

I had some free time today so I tried to spin up the server again. Here is the log from my own coturn server:

Note that my private ip address range is 10.0.0.0/8, so I am not blocking those IPs specifically in my denied ip list. And 

0: (1): INFO: System cpu num is 20
0: (1): INFO: System cpu num is 20
0: (1): INFO: System enable num is 20
0: (1): INFO: Listener address to use: 0.0.0.0
0: (1): INFO: Black listing: 192.168.0.0-192.168.255.255
0: (1): INFO: Black listing: 172.16.0.0-172.31.255.255
0: (1): INFO: Black listing: 0.0.0.0-0.255.255.255
0: (1): INFO: Black listing: 100.64.0.0-100.127.255.255
0: (1): INFO: Black listing: 127.0.0.0-127.255.255.255
0: (1): INFO: Black listing: 169.254.0.0-169.254.255.255
0: (1): INFO: Black listing: 192.0.0.0-192.0.0.255
0: (1): INFO: Black listing: 192.0.2.0-192.0.2.255
0: (1): INFO: Black listing: 192.88.99.0-192.88.99.255
0: (1): INFO: Black listing: 198.18.0.0-198.19.255.255
0: (1): INFO: Black listing: 198.51.100.0-198.51.100.255
0: (1): INFO: Black listing: 203.0.113.0-203.0.113.255
0: (1): INFO: Black listing: 240.0.0.0-255.255.255.255
0: (1): ERROR: You cannot define external IP more than once in the configuration
0: (1): INFO: Coturn Version Coturn-4.6.2 'Gorst'
0: (1): INFO: Max number of open files/sockets allowed for this process: 1073741816
0: (1): INFO: Due to the open files/sockets limitation, max supported number of TURN Sessions possible is: 536870500 (approximately)
0: (1): INFO:

==== Show him the instruments, Practical Frost: ====

0: (1): INFO: OpenSSL compile-time version: OpenSSL 1.1.1n  15 Mar 2022 (0x101010ef)
0: (1): INFO: TLS 1.3 supported
0: (1): INFO: DTLS 1.2 supported
0: (1): INFO: TURN/STUN ALPN supported
0: (1): INFO: Third-party authorization (oAuth) supported
0: (1): INFO: GCM (AEAD) supported
0: (1): INFO: SQLite supported, default database location is /var/lib/coturn/turndb
0: (1): INFO: Redis supported
0: (1): INFO: PostgreSQL supported
0: (1): INFO: MySQL supported
0: (1): INFO: MongoDB supported
0: (1): INFO: Default Net Engine version: 3 (UDP thread per CPU core)
0: (1): INFO: Domain name:
0: (1): INFO: Default realm: xxxx.com
0: (1): ERROR: CONFIG: Empty cli-password, and so telnet cli interface is disabled! Please set a non empty cli-password!
0: (1): INFO: Certificate file found: /cert/live/xxxx.com/fullchain.pem
0: (1): INFO: Private key file found: /cert/live/xxxx.com/privkey.pem
0: (1): INFO: TLS cipher suite: DEFAULT:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_128_GCM_SHA256
0: (1): INFO: DTLS cipher suite: DEFAULT:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_128_GCM_SHA256
0: (1): INFO: Relay address to use: 0.0.0.0
0: (1): INFO: pid file created: /var/run/turnserver.pid
0: (1): INFO: IO method: epoll (with changelist)
0: (1): WARNING: STUN CHANGE_REQUEST not supported: only one IP address is provided
0: (1): INFO: Wait for relay ports initialization...
0: (1): INFO:   relay 0.0.0.0 initialization...
0: (1): INFO:   relay 0.0.0.0 initialization done
0: (1): INFO: Relay ports initialization done
0: (37): DEBUG: turn server id=4 created
0: (35): DEBUG: turn server id=2 created
0: (36): DEBUG: turn server id=3 created
0: (37): INFO: IPv4. TLS/TCP listener opened on : 0.0.0.0:3478
0: (35): INFO: IPv4. TLS/TCP listener opened on : 0.0.0.0:3478
0: (36): INFO: IPv4. TLS/TCP listener opened on : 0.0.0.0:3478
0: (40): DEBUG: turn server id=7 created
0: (38): DEBUG: turn server id=5 created
0: (38): INFO: IPv4. TLS/TCP listener opened on : 0.0.0.0:3478
0: (37): INFO: IPv4. TLS/TCP listener opened on : 0.0.0.0:5349
0: (35): INFO: IPv4. TLS/TCP listener opened on : 0.0.0.0:5349
0: (38): INFO: IPv4. TLS/TCP listener opened on : 0.0.0.0:5349
0: (40): INFO: IPv4. TLS/TCP listener opened on : 0.0.0.0:3478
0: (34): DEBUG: turn server id=1 created
0: (40): INFO: IPv4. TLS/TCP listener opened on : 0.0.0.0:5349
0: (34): INFO: IPv4. TLS/TCP listener opened on : 0.0.0.0:3478
0: (36): INFO: IPv4. TLS/TCP listener opened on : 0.0.0.0:5349
0: (34): INFO: IPv4. TLS/TCP listener opened on : 0.0.0.0:5349
0: (33): DEBUG: turn server id=0 created
0: (33): INFO: IPv4. TLS/TCP listener opened on : 0.0.0.0:3478
0: (39): DEBUG: turn server id=6 created
0: (33): INFO: IPv4. TLS/TCP listener opened on : 0.0.0.0:5349
0: (39): INFO: IPv4. TLS/TCP listener opened on : 0.0.0.0:3478
0: (39): INFO: IPv4. TLS/TCP listener opened on : 0.0.0.0:5349
0: (41): DEBUG: turn server id=8 created
0: (41): INFO: IPv4. TLS/TCP listener opened on : 0.0.0.0:3478
0: (41): INFO: IPv4. TLS/TCP listener opened on : 0.0.0.0:5349
0: (42): DEBUG: turn server id=9 created
0: (42): INFO: IPv4. TLS/TCP listener opened on : 0.0.0.0:3478
0: (42): INFO: IPv4. TLS/TCP listener opened on : 0.0.0.0:5349
0: (44): DEBUG: turn server id=11 created
0: (44): INFO: IPv4. TLS/TCP listener opened on : 0.0.0.0:3478
0: (44): INFO: IPv4. TLS/TCP listener opened on : 0.0.0.0:5349
0: (43): DEBUG: turn server id=10 created
0: (43): INFO: IPv4. TLS/TCP listener opened on : 0.0.0.0:3478
0: (43): INFO: IPv4. TLS/TCP listener opened on : 0.0.0.0:5349
0: (45): DEBUG: turn server id=12 created
0: (45): INFO: IPv4. TLS/TCP listener opened on : 0.0.0.0:3478
0: (45): INFO: IPv4. TLS/TCP listener opened on : 0.0.0.0:5349
0: (46): DEBUG: turn server id=13 created
0: (46): INFO: IPv4. TLS/TCP listener opened on : 0.0.0.0:3478
0: (47): DEBUG: turn server id=14 created
0: (46): INFO: IPv4. TLS/TCP listener opened on : 0.0.0.0:5349
0: (47): INFO: IPv4. TLS/TCP listener opened on : 0.0.0.0:3478
0: (47): INFO: IPv4. TLS/TCP listener opened on : 0.0.0.0:5349
0: (48): DEBUG: turn server id=15 created
0: (49): DEBUG: turn server id=16 created
0: (49): INFO: IPv4. TLS/TCP listener opened on : 0.0.0.0:3478
0: (48): INFO: IPv4. TLS/TCP listener opened on : 0.0.0.0:3478
0: (49): INFO: IPv4. TLS/TCP listener opened on : 0.0.0.0:5349
0: (48): INFO: IPv4. TLS/TCP listener opened on : 0.0.0.0:5349
0: (50): DEBUG: turn server id=17 created
0: (51): DEBUG: turn server id=18 created
0: (50): INFO: IPv4. TLS/TCP listener opened on : 0.0.0.0:3478
0: (51): INFO: IPv4. TLS/TCP listener opened on : 0.0.0.0:3478
0: (50): INFO: IPv4. TLS/TCP listener opened on : 0.0.0.0:5349
0: (51): INFO: IPv4. TLS/TCP listener opened on : 0.0.0.0:5349
0: (1): INFO: Total General servers: 20
0: (52): DEBUG: turn server id=19 created
0: (52): INFO: IPv4. TLS/TCP listener opened on : 0.0.0.0:3478
0: (52): INFO: IPv4. TLS/TCP listener opened on : 0.0.0.0:5349
0: (1): INFO: IPv4. DTLS/UDP listener opened on: 0.0.0.0:3478
0: (1): INFO: IPv4. DTLS/UDP listener opened on: 0.0.0.0:5349
0: (1): INFO: Total auth threads: 11
0: (1): INFO: prometheus collector disabled, not started
21: (35): INFO: IPv4. tcp or tls connected to: 104.223.93.152:39033 (This is Windscribe VPN that I connected to)
21: (35): INFO: session 002000000000000001: realm <xxxx.com> user <>: incoming packet message processed, error 401: Unauthorized
21: (35): INFO: IPv4. Local relay addr: 10.0.0.2:49158 (This is my server behind NAT in my home)
21: (35): INFO: session 002000000000000001: new, realm=<xxxx.com>, username=<0123456789:@bbbb:xxxx.com>, lifetime=600, cipher=TLS_AES_256_GCM_SHA384, method=TLSv1.3
21: (35): INFO: session 002000000000000001: realm <xxxx.com> user <0123456789:@bbbb:xxxx.com>: incoming packet ALLOCATE processed, success
21: (35): INFO: session 002000000000000001: peer 10.0.0.3 lifetime updated: 300
21: (35): INFO: session 002000000000000001: realm <xxxx.com> user <0123456789:@bbbb:xxxx.com>: incoming packet CREATE_PERMISSION processed, success
36: (42): INFO: IPv4. tcp or tls connected to: 104.223.93.152:40963
36: (42): INFO: session 009000000000000001: realm <xxxx.com> user <>: incoming packet message processed, error 401: Unauthorized
36: (42): INFO: IPv4. Local relay addr: 10.0.0.2:49167
36: (42): INFO: session 009000000000000001: new, realm=<xxxx.com>, username=<0123456789:@bbbb:xxxx.com>, lifetime=600, cipher=TLS_AES_256_GCM_SHA384, method=TLSv1.3
36: (42): INFO: session 009000000000000001: realm <xxxx.com> user <0123456789:@bbbb:xxxx.com>: incoming packet ALLOCATE processed, success
36: (42): INFO: session 009000000000000001: peer 10.0.0.3 lifetime updated: 300
36: (42): INFO: session 009000000000000001: realm <xxxx.com> user <0123456789:@bbbb:xxxx.com>: incoming packet CREATE_PERMISSION processed, success
82: (42): INFO: session 009000000000000001: refreshed, realm=<xxxx.com>, username=<0123456789:@bbbb:xxxx.com>, lifetime=0, cipher=TLS_AES_256_GCM_SHA384, method=TLSv1.3
82: (35): INFO: session 002000000000000001: refreshed, realm=<xxxx.com>, username=<0123456789:@bbbb:xxxx.com>, lifetime=0, cipher=TLS_AES_256_GCM_SHA384, method=TLSv1.3
82: (42): INFO: session 009000000000000001: realm <xxxx.com> user <0123456789:@bbbb:xxxx.com>: incoming packet REFRESH processed, success
82: (35): INFO: session 002000000000000001: realm <xxxx.com> user <0123456789:@bbbb:xxxx.com>: incoming packet REFRESH processed, success
82: (42): INFO: session 009000000000000001: TLS/TCP socket disconnected: 104.223.93.152:40963
82: (42): INFO: session 009000000000000001: usage: realm=<xxxx.com>, username=<0123456789:@bbbb:xxxx.com>, rp=82, rb=10724, sp=4, sb=348
82: (42): INFO: session 009000000000000001: peer usage: realm=<xxxx.com>, username=<0123456789:@bbbb:xxxx.com>, rp=0, rb=0, sp=78, sb=7488
82: (42): INFO: session 009000000000000001: closed (2nd stage), user <0123456789:@bbbb:xxxx.com> realm <xxxx.com> origin <>, local 0.0.0.0:5349, remote 104.223.93.152:40963, reason: TLS/TCP socket buffer operation error (callback)
82: (42): INFO: session 009000000000000001: delete: realm=<xxxx.com>, username=<0123456789:@bbbb:xxxx.com>
82: (42): INFO: session 009000000000000001: peer 10.0.0.3 deleted
82: (35): INFO: session 002000000000000001: TLS/TCP socket disconnected: 104.223.93.152:39033
82: (35): INFO: session 002000000000000001: usage: realm=<xxxx.com>, username=<0123456789:@bbbb:xxxx.com>, rp=323, rb=39848, sp=88, sb=13788
82: (35): INFO: session 002000000000000001: peer usage: realm=<xxxx.com>, username=<0123456789:@bbbb:xxxx.com>, rp=84, rb=8400, sp=319, sb=27936
82: (35): INFO: session 002000000000000001: closed (2nd stage), user <0123456789:@bbbb:xxxx.com> realm <xxxx.com> origin <>, local 0.0.0.0:5349, remote 104.223.93.152:39033, reason: TLS/TCP socket buffer operation error (callback)
82: (35): INFO: session 002000000000000001: delete: realm=<xxxx.com>, username=<0123456789:@bbbb:xxxx.com>
82: (35): INFO: session 002000000000000001: peer 10.0.0.3 deleted
J4gQBqqR commented 1 year ago

My coturn config (using valid zeroSSL cert):

use-auth-secret
static-auth-secret=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
realm=xxxx.com

listening-port=3478
tls-listening-port=5349
min-port=49152
max-port=49172
external-ip=xx.xxx.xx.xxx (This is the external IP of my Home)
listening-ip=0.0.0.0

# don't let the relay ever try to connect to private IP address ranges within your network (if any)
# given the turn server is likely behind your firewall, remember to include any privileged public IPs too.
#denied-peer-ip=10.0.0.0-10.255.255.255
denied-peer-ip=192.168.0.0-192.168.255.255
denied-peer-ip=172.16.0.0-172.31.255.255

# recommended additional local peers to block, to mitigate external access to internal services.
# https://www.rtcsec.com/article/slack-webrtc-turn-compromise-and-bug-bounty/#how-to-fix-an-open-turn-relay-to-address-this-vulnerability
no-multicast-peers
denied-peer-ip=0.0.0.0-0.255.255.255
denied-peer-ip=100.64.0.0-100.127.255.255
denied-peer-ip=127.0.0.0-127.255.255.255
denied-peer-ip=169.254.0.0-169.254.255.255
denied-peer-ip=192.0.0.0-192.0.0.255
denied-peer-ip=192.0.2.0-192.0.2.255
denied-peer-ip=192.88.99.0-192.88.99.255
denied-peer-ip=198.18.0.0-198.19.255.255
denied-peer-ip=198.51.100.0-198.51.100.255
denied-peer-ip=203.0.113.0-203.0.113.255
denied-peer-ip=240.0.0.0-255.255.255.255

# VoIP traffic is all UDP. There is no reason to let users connect to arbitrary TCP endpoints via the relay.
#no-tcp-relay

# special case the turn server itself so that client->TURN->TURN->client flows work
#allowed-peer-ip=10.0.0.1-10.0.0.2
#allowed-peer-ip=0.0.0.0-255.255.255.255

# consider whether you want to limit the quota of relayed streams per user (or total) to avoid risk of DoS.
user-quota=12 # 4 streams per video call, so 12 streams = 3 simultaneous relayed calls per user.
total-quota=1200

# TLS certificates, including intermediate certs.
# For Let's Encrypt certificates, use `fullchain.pem` here.
cert=/cert/live/xxxx.com/fullchain.pem (I doubled checked my cert, is zeroSSL cert issued on March 12, valid cert)

# TLS private key file
pkey=/cert/live/xxxx.com/privkey.pem

#no-tls
#no-dtls

verbose=4