thestinger
commented
11 months ago This keeps happening for the GrapheneOS rooms, which were among the largest Matrix communities until this happened. It's always the same symptom of join rules, power levels, etc. resetting and then all users who joined since the join rules it resets to being dropped out of the room with the room bricked in Element where they can't send, receive or leave without clearing cache.
Description
I have encountered yet another room state reset.
I suspect that the reset of
m.room.join_ruleswas the root cause of the 51m.room.memberresets, as the latter would now be rejected events. There is no obvious explanation for them.room.join_rulesreset.No state events were sent or redacted in the time immediately preceding the state reset. The last state event before the reset was sent approximately 2 hours prior.
Steps to reproduce
Homeserver
arcticfoxes.net
Synapse Version
1.96.0rc1
Installation Method
Debian packages from packages.matrix.org
Database
PostgreSQL 15.4, single server, not ported, restored a very long time ago
Workers
Single process
Platform
Configuration
Relevant log output
``` stream_id | room_id | type | state_key | event_id | prev_event_id | instance_name -----------+-------------------------------------+-------------------+-------------------------------------+----------------------------------------------+----------------------------------------------+--------------- 25794806 | !frWBWOPyriepLexnsA:arcticfoxes.net | m.room.member | @aldorithms:matrix.org | | $74_XqgpHw_D18w0Xq0Lrc7_nmBvzDx1UajEn9yKc3Qc | master 25794806 | !frWBWOPyriepLexnsA:arcticfoxes.net | m.room.member | @jhs177992f:arcticfoxes.net | | $rFYpIiFmMMFHYqY_835sCFOEpJVUyLpEDM5Xoqr4uPc | master 25794806 | !frWBWOPyriepLexnsA:arcticfoxes.net | m.room.member | @ariyaro:matrix.org | | $JqoI5tf1YrjoifsPEdxT67IeLJW6BCAdhIsFgxIJQ9Y | master 25794806 | !frWBWOPyriepLexnsA:arcticfoxes.net | m.room.member | @andreus:matrix.org | | $CUryaK8LXBZxMsDlVohrahm3LaiBC-uCoHHOot0UWS4 | master 25794806 | !frWBWOPyriepLexnsA:arcticfoxes.net | m.room.member | @heliopolis:nitro.chat | | $utSCExwz0EzMAD5eIpCZ9BeEcinoslMFKTjgLO-H6rU | master 25794806 | !frWBWOPyriepLexnsA:arcticfoxes.net | m.room.member | @backpack:matrix.org | | $cCS3Ry7EFHCXQHTyMrJLratjz0GYRGsyCqU5AkDcnQ4 | master 25794806 | !frWBWOPyriepLexnsA:arcticfoxes.net | m.room.member | @tfox42:matrix.org | | $93-srvPW852lSXze_r-y5VsDsi32Jf6H0GbXxrI8zFo | master 25794806 | !frWBWOPyriepLexnsA:arcticfoxes.net | m.room.member | @houndvoyager:matrix.org | | $5XMMvwgqdHpX6eyBzamDKpTADycXXvdUhMAIKhtPbTY | master 25794806 | !frWBWOPyriepLexnsA:arcticfoxes.net | m.room.member | @mercifulrage:converser.eu | | $EDFgLxKC6dz2cG7BYk7gO5xY2NNIKKRcMr7bM9XrPyA | master 25794806 | !frWBWOPyriepLexnsA:arcticfoxes.net | m.room.member | @crypticangel:pragma-messenger.ch | | $QG5tHgkzolxZAS0E3auilaPTjZqssp5fFr3NPgiwex0 | master 25794806 | !frWBWOPyriepLexnsA:arcticfoxes.net | m.room.member | @coalminecanary:arcticfoxes.net | | $SG_cN3tHXhg9_7_mRHe5Qt4wFlQO7o8hvoV1XPepkTA | master 25794806 | !frWBWOPyriepLexnsA:arcticfoxes.net | m.room.member | @vovafreeman:matrix.org | | $r9YGaq9q9dbye19fLlO09TO-IC9k6bfOWl-unkJRNNw | master 25794806 | !frWBWOPyriepLexnsA:arcticfoxes.net | m.room.member | @qoijjj:arcticfoxes.net | | $XbGI2ynAn3WC5hGRrmsBmRbW6f3l5mzzyXoDsqp_C00 | master 25794806 | !frWBWOPyriepLexnsA:arcticfoxes.net | m.room.member | @mebious:tchncs.de | | $zYy_smuirJ0TDeYdC5vgbyQQRxHqxTTF9OTdQ1-cjhs | master 25794806 | !frWBWOPyriepLexnsA:arcticfoxes.net | m.room.member | @windystone:matrix.org | | $Qa-UYRrnev1cof2voq4VrjlmnDmvEn7cF6QgIYomxFA | master 25794806 | !frWBWOPyriepLexnsA:arcticfoxes.net | m.room.member | @user0761:matrix.org | | $4w8S40ugloh8ixEufM0tj-_QvnR3oaiXXqdBIPicZDg | master 25794806 | !frWBWOPyriepLexnsA:arcticfoxes.net | m.room.member | @matrixruinedmylife:arcticfoxes.net | | $Q0mg0Le1GBwXh_H2tFLvwCM97Kf-njC0tW5GcqyFVPU | master 25794806 | !frWBWOPyriepLexnsA:arcticfoxes.net | m.room.member | @suicidalprincess:nitro.chat | | $Jv4B2FBIDgxkMR93mc9CNE88aTJqG8NrB79tMeVqhJY | master 25794806 | !frWBWOPyriepLexnsA:arcticfoxes.net | m.room.member | @hat:arcticfoxes.net | | $IDqAum_6ytZX_u0tz-8JglUBjwD39edl329bmExXrzo | master 25794806 | !frWBWOPyriepLexnsA:arcticfoxes.net | m.room.member | @iustitia:4d2.org | | $86lLB5SWm9xB93vZroSDU4hfjGQ-ZYrJr9fbnE2DZRE | master 25794806 | !frWBWOPyriepLexnsA:arcticfoxes.net | m.room.member | @beyiyaf450:matrix.org | | $DgRnGVumPcveQGtNWGLKrMzBwowHRJwtrRggmA_ndFo | master 25794806 | !frWBWOPyriepLexnsA:arcticfoxes.net | m.room.member | @fpmksoba:matrix.org | | $2GtIIACc89vvMrYb1AKJeF0ky4IgDtsZpsFCy_trBTM | master 25794806 | !frWBWOPyriepLexnsA:arcticfoxes.net | m.room.member | @sneber:nope.chat | | $9mpOqALVoUK1SQ6BbV4H0wKaiooTXbbHnmAIyDjXlvw | master 25794806 | !frWBWOPyriepLexnsA:arcticfoxes.net | m.room.member | @austin:tchncs.de | | $zg1FQYQk3pTix5UOY7PRxSCmE6Q-94lbXnpgMbByoys | master 25794806 | !frWBWOPyriepLexnsA:arcticfoxes.net | m.room.member | @bhydden:matrix.org | | $tp1MSrvb4KOEPtxqtJNQ6bKRMdzOVs7puBihkkI6d_4 | master 25794806 | !frWBWOPyriepLexnsA:arcticfoxes.net | m.room.member | @masterrank:matrix.org | | $GJ4SZsaZuJg0NOS0GauO9DcNX3Qa1ESTSP-C84YbSYQ | master 25794806 | !frWBWOPyriepLexnsA:arcticfoxes.net | m.room.member | @w14:matrix.org | | $i-ZGHSTUJuRLjw6y7IZVSxAbxYZFi42AgYK_xkzEeOA | master 25794806 | !frWBWOPyriepLexnsA:arcticfoxes.net | m.room.member | @cyri:matrix.org | | $hxptXTV6_m2O4ZoP0w69HOIc1FQ22GC-UnJUz6d0HwQ | master 25794806 | !frWBWOPyriepLexnsA:arcticfoxes.net | m.room.member | @ilove9999:matrix.org | | $k3C1PKERqlnX3ulqHe5qzpzZKfTVWQrknX1z5avD2Dc | master 25794806 | !frWBWOPyriepLexnsA:arcticfoxes.net | m.room.member | @quietlyjaded:matrix.org | | $NRY6fXAazYFaVa0kLpYWIhy2gfqJW8tVCFyZxVul01E | master 25794806 | !frWBWOPyriepLexnsA:arcticfoxes.net | m.room.member | @green_sheep:matrix.org | | $cGwRLAiQxivw7d3h9uMca5egIkDOhRVgZnD2Gl_zWVk | master 25794806 | !frWBWOPyriepLexnsA:arcticfoxes.net | m.room.member | @tisktisk:monero.social | | $lZ01gtBU50UhZ8sacAtpp6Ia5h6Gd54jON7o6Rxu9mc | master 25794806 | !frWBWOPyriepLexnsA:arcticfoxes.net | m.room.member | @loliloli:arcticfoxes.net | | $iPSlyT37ecv2ZqGH4yOAhkJ6_C51_J-muduqoeSf7PU | master 25794806 | !frWBWOPyriepLexnsA:arcticfoxes.net | m.room.member | @chili1803:matrix.org | | $jUwmF9DuNmdb56KzAP_D364QM7aocojUY-O95j6hx0c | master 25794806 | !frWBWOPyriepLexnsA:arcticfoxes.net | m.room.member | @sclanchy:matrix.org | | $27hLXs_qKe4WIgGqy6GigIV6TuknsBmzDSgBHn4c3TI | master 25794806 | !frWBWOPyriepLexnsA:arcticfoxes.net | m.room.member | @faxex:matrix.org | | $tz3qjuYJnJTHQBaUotnXXVQOEeoUAyGIQTd-1sdhKXc | master 25794806 | !frWBWOPyriepLexnsA:arcticfoxes.net | m.room.member | @ghr89sb:matrix.org | | $UgsO3HsBuDnYVpxzpda-7WUH5vh9AujnN7dEcXX7L5A | master 25794806 | !frWBWOPyriepLexnsA:arcticfoxes.net | m.room.member | @poisonivy:arcticfoxes.net | | $pVH8QOPHfNfuv1cILOOSELlOuLAPKbliYYbflMkOg-g | master 25794806 | !frWBWOPyriepLexnsA:arcticfoxes.net | m.room.member | @johan942:matrix.org | | $jjKcZyb7w2DcbYvaKWf5c_052nutBDxcMbM3FJI5V5o | master 25794806 | !frWBWOPyriepLexnsA:arcticfoxes.net | m.room.member | @carlfriedrich:tchncs.de | | $CeMBICUBs6UPB1hpKooo7YjiTsbevuzC7IRjxihiOxU | master 25794806 | !frWBWOPyriepLexnsA:arcticfoxes.net | m.room.member | @2nami:matrix.org | | $MEz7DkYNjmX1dXW4aJoiGN8kkNnQ1zjmGekNh3fQijU | master 25794806 | !frWBWOPyriepLexnsA:arcticfoxes.net | m.room.member | @un32:matrix.org | | $JAAIV6FTLNgANKHyi1dhxskos-RngW_0tr44roqbRL4 | master 25794806 | !frWBWOPyriepLexnsA:arcticfoxes.net | m.room.member | @ancloadi:matrix.org | | $Dw7r7MLjd3JqLzzVdHKuXe0eVsO4wXQzNpOUvoCGtAQ | master 25794806 | !frWBWOPyriepLexnsA:arcticfoxes.net | m.room.member | @losintos:matrix.org | | $Viw3Yw_2fyjF6Jnk4-ozYMOq6JWrt2TasbEhagy7jyE | master 25794806 | !frWBWOPyriepLexnsA:arcticfoxes.net | m.room.member | @xyhhx:arcticfoxes.net | | $F6xBiS5WgeyYB7-R4B8eOMKWOD9LA0yNjGj_mDDJEX0 | master 25794806 | !frWBWOPyriepLexnsA:arcticfoxes.net | m.room.join_rules | | $AxE3gEzTLPT_hy7ML1MJKsX7DI6XkqujWP50LTDBODM | $P2FwR8kpzW1LDIZU2lKRc_wmghbXlIQZpwvE5D82d5Q | master 25794806 | !frWBWOPyriepLexnsA:arcticfoxes.net | m.room.member | @dahustlah:matrix.org | $8GW8hJvAHAGPTQIsPysF_mePLpd-GtZyqYIb4VzfoiU | $aGradtN72do5IOaG1dvRExtW4PAwFfjdlTNZVyOP6sQ | master 25794806 | !frWBWOPyriepLexnsA:arcticfoxes.net | m.room.member | @gvl7bcq6dn2xrbl:matrix.org | $_s6-w6-v0tE3-tYkWYQ1bbukUs844bn_9bUkttYCXF0 | $iNV97F-d1DnI0dGqj7L23a2pTtRmvSCNPpfDfGbGbg4 | master 25794806 | !frWBWOPyriepLexnsA:arcticfoxes.net | m.room.member | @masayuki:wired.rehab | $yMVmIFtT_hpeMh-9GLSdY84NrNYSRhn8rUX2Z2AD5_g | $egJIEy0PhxdUfdDCAPAVJDSrIYTLYtdAIIWCQ3zvsNA | master 25794806 | !frWBWOPyriepLexnsA:arcticfoxes.net | m.room.member | @matchboxbananasynergy:matrix.org | $A4R_VVl2TaN53EWiLBoipPyigkEZodgWOiceeieshaQ | $s-1rvX3ldLsL5Azxj7mSjkhWxboqaEuQiJlMCglMIYk | master 25794806 | !frWBWOPyriepLexnsA:arcticfoxes.net | m.room.member | @skull_gap:archoslinux.cz | $MX2EhPV0TYzMxegHKJX1U1KKvqTrAp8vwS5DwqqDlNw | $_0SGO3HGR2RgYWYAxczIkwXzCl7e3PLuZgrO_HTgj6Y | master 25794806 | !frWBWOPyriepLexnsA:arcticfoxes.net | m.room.member | @slate-module:matrix.org | $xM6Y-mUyvW9yOZf_lODJOdqLI0Ao1nYuaYQYonysi84 | $2IUW5LONVOx_k1I5nxRv9AuF8BFAp5NL8I_KxFa16Zk | master (52 rows) ```SELECT * FROM current_state_delta_stream WHERE room_id='!frWBWOPyriepLexnsA:arcticfoxes.net' AND stream_id=25794806;Anything else that would be useful to know?
In addition to arcticfoxes.net documented above, this state reset was confirmed via the C‑S API on matrix.org (Synapse 1.96.0rc1) and a third server (Synapse 1.95.1). I do not know the exact details of their deployments.