Missing record data in messages from relay

[mackuba]

Describe the bug

I've seen this happen once or twice before. I got a bunch of errors in the firehose client last night, about missing record data in a post create event. The posts come from this account https://bsky.app/profile/did:plc:h52iup5ewvpkm2oaeipdmxgt, which is obviously doing something non-standard (the posts seem to be missing on the website). The previous case was also with some kind of account that looked like it was importing posts from Twitter.

The messages from the relay were between 2023-12-11 08:23:48 UTC and 08:24:03, seq numbers between 434007654 and 434008353.

Example sections content parsed from message 434007654, which creates a post record app.bsky.feed.post/3kgaynnerhz2l:

{"did":"did:plc:h52iup5ewvpkm2oaeipdmxgt","rev":"3kgaynngk4v22","sig":{"$bytes":"lFtFk2NgSFbYi3I9SqYqaNF149Gh3LkHwVZ5jtDXrDpKcd8Z9cjzG4ig+Bwu\ni+BbhD/bAW/f2vNOc0L7nRPUHQ"},"data":{"$link":"bafyreievw7mjecplu5isujki55f4elnmliowyshh7mrvhasofsnfubcpdm"},"prev":null,"version":3}

{"e":[{"k":{"$bytes":"YXBwLmJza3kuZmVlZC5wb3N0LzNrZnVwem5zc2R0MmE"},"p":0,"t":{"$link":"bafyreifr3omymk3b67al4syv56xhijqh7lwyh3jcwatoxgptjnks2zt7pu"},"v":{"$link":"bafyreiepae3surj3abirqtb2hnnv4s7grpmabzzswj4htb22vuxzj4hvyi"}},{"k":{"$bytes":"empveDczNDJm"},"p":23,"t":{"$link":"bafyreigrijbsddidoqoq2gkbx6oqg4ulyjkus4ujmysjhdtd2vxwclqqdm"},"v":{"$link":"bafyreicj465oco3qv3uebnba7nk5e2unijtcjleoplmf2bnqzzayzwhmxm"}},{"k":{"$bytes":"eWN6am9wMjMycA"},"p":22,"t":{"$link":"bafyreiglzrmfgxdaw56rkbdhwjrpngubntrrk4auiipr7lv7hai45632ae"},"v":{"$link":"bafyreiegbalmtgjy2ucwrtzch5oatxbiubbsg3nvc36lnb65e3tv66yqve"}},{"k":{"$bytes":"Z2F5bmV6ZXRkMnM"},"p":21,"t":{"$link":"bafyreicxj6uakwc2r7d5sdwdm5hsohwqqpcxybaagpzqmjlja2qrkw3rhm"},"v":{"$link":"bafyreicactlmkfw5dw275rvyqcfp72tcwdpfhorzfz62a6w76ocvidegia"}}],"l":{"$link":"bafyreiekjuqahmrdevjmfalbebusp3z7jdvxznejomujadmvv6hr5blfsi"}}

{"e":[],"l":{"$link":"bafyreiavauw3ydac447vlmxb3nxvfgvpvqgl4tmizmb6ymijw6ucjrafvy"}}    

{"e":[{"k":{"$bytes":"YXBwLmJza3kuZmVlZC5wb3N0LzNrZ2F5bmx2Yng2MmU"},"p":0,"t":{"$link":"bafyreicwjqwmleshfxy22zszdt7ulbdv2n4ttxidvpqnz37odhz3ef55k4"},"v":{"$link":"bafyreigo4tkp2uspsyy4qousjldzbn5ojeqdphwi27fauqqztmwj2u7gk4"}}],"l":{"$link":"bafyreidzzond24fdmu3lfrfngop4c2etdz2mn3ys2zlylitvta5sp6lprm"}}

{"e":[],"l":{"$link":"bafyreidkjyt3vcoj4b4ak65vxfoyleakl75tiznevefrsmtw6aqfxglsx4"}}    

{"e":[],"l":{"$link":"bafyreifa5efymuczhazrzjmgnlix3bhyij5cgohmwsndfpnb2y7ura6une"}}

{"e":[{"k":{"$bytes":"YXBwLmJza3kuZmVlZC5wb3N0LzNrZ2F5bm03ZzV3MnY"},"p":0,"t":{"$link":"bafyreifxl46i5chjsqibzzqwkdntv6ujpmujfqivzwkrsg7447cijrvgfy"},"v":{"$link":"bafyreihzaf376a5sjpy2ngrphrinedemtdkwqvgm4v6hppok6lut7vdnny"}},{"k":{"$bytes":"bGp1MzJp"},"p":26,"t":{"$link":"bafyreihiufxwxk5rglv4a6bhjr7hbzhd4rnxqpasz7d32jcmg5iwylfa6a"},"v":{"$link":"bafyreifge6cw7mzpixcrfhfauqyxu3dabr46rh3cd7pvyqkatbkjqztbpm"}},{"k":{"$bytes":"bjRmdmoybA"},"p":25,"t":null,"v":{"$link":"bafyreicyrar3yga4oubmimjmenimfxmvnbsmupvytkobiz5pdz776qz6xi"}},{"k":{"$bytes":"NTVkaTJ5"},"p":26,"t":{"$link":"bafyreiglgzpqiqiww5vgcbb6bpkxeq2bgt3oy23opwbhm6uibxdng6ljy4"},"v":{"$link":"bafyreihlwzb5pubgukuomf4frwql6y6idwcl3osjaypkuiv2u4z6uchtqu"}},{"k":{"$bytes":"YXdnczJi"},"p":26,"t":{"$link":"bafyreiaigvv7tvway7aenqc7slbo4hmjs23zca4w2sbcagpwbuhfdbnluy"},"v":{"$link":"bafyreihjxreqa7llo6u3enzgs564ryqzcncuxauopicc37xwrzok25lmwq"}}],"l":{"$link":"bafyreica3ow6b5bkfrelsoigb4ne4qsdni2yq76up2mh6hwtkadligvyrm"}}

{"e":[{"k":{"$bytes":"YXBwLmJza3kuZmVlZC5wb3N0LzNrZ2F5bm5jNmhuMnc"},"p":0,"t":{"$link":"bafyreie3hcsh6esiiqwhuik4qp63v7udlidpqwkjrxvmy7p4dbupvvrjrm"},"v":{"$link":"bafyreidpxxqo6srqxsaeagyeumt2ypt2aoamp7ba7q5y7yllhphi3qrihy"}},{"k":{"$bytes":"ZGRsbTJ1"},"p":26,"t":{"$link":"bafyreicxrvgt5bqm47tpmbkvssdsgktl2nnap2algwlpjptzsqg2kualtq"},"v":{"$link":"bafyreicz77fbu5lfzxy45fw27dhdtqrbfsze2zil6slhevo34mbbb6ymji"}},{"k":{"$bytes":"Z3JhcGguYmxvY2svM2tnM3U3dWxnajUybQ"},"p":9,"t":{"$link":"bafyreifqfx7o6k5sdmzqmrnksnunbusjtd4irtt6ml4zgrgxwjffw2cwam"},"v":{"$link":"bafyreid3cud5moa7oxxt7ok3y5tuocqllyhfholcchzoectutgvchb5oaq"}},{"k":{"$bytes":"YjdxcW51Mjc"},"p":26,"t":{"$link":"bafyreiapd6adgjzssya6j5y5jbyr745qan4jvitzyakcrsbfwozbipxdau"},"v":{"$link":"bafyreid4lvyrkz23fsjkgavizpotnx3xute74wcbboskyerr4j4o4vdnt4"}},{"k":{"$bytes":"aG9oeDZrMmw"},"p":26,"t":{"$link":"bafyreihrgufj2h4wveu7c466agaipht3db37j4psn3kyu5o7na3pct4fyi"},"v":{"$link":"bafyreibm73qdlni222x7lii72coh2epirlettebek3h5l6qjvloyyb7goy"}},{"k":{"$bytes":"aXhicWR6MnU"},"p":26,"t":{"$link":"bafyreic4fauoglttdisyr6fouku6jti3utbunp6b52pqw33fsbdywyunty"},"v":{"$link":"bafyreihiirslcfefcnn6k3cuwdwcyq2afxiu4lmrb7qcekdnwucy4howty"}},{"k":{"$bytes":"bmJsNW41MjI"},"p":26,"t":null,"v":{"$link":"bafyreieqk4rrurbk5fby4i7sactrutti3z5ozqurvk3ex26y7f5xqlf2ea"}},{"k":{"$bytes":"eWhna3d2MjM"},"p":26,"t":{"$link":"bafyreid5u2uwwuhbqzyfd6lakldj75zftu2mbzhimql4l6j4mxry3f2hgq"},"v":{"$link":"bafyreib2ewmnya523c2vjxqbr4gqmvqmusywzs45wdq7pttsacyybpjlza"}},{"k":{"$bytes":"Zm9sbG93LzNrZnhuenc0NzJ0Mms"},"p":15,"t":{"$link":"bafyreialb3dvnsxf3e43lhdko64ussa65xg72njn5zcfhjbe3v4mr5lllq"},"v":{"$link":"bafyreifq37hgfmlw77jfnfjf7mvnykqq2brjge4mml5klru6ml5hgb36pe"}},{"k":{"$bytes":"bzQ1N3ZkYzJs"},"p":26,"t":{"$link":"bafyreiaq5gdwj5mlpyt3zoschcuff3jk6vanne6blwwersu2f53mg552km"},"v":{"$link":"bafyreieoxsk4vc2ifst3azlfupmsalrpne2mjsft32etatntigpyxvpgly"}},{"k":{"$bytes":"NXI3NHhsMnY"},"p":27,"t":{"$link":"bafyreifhltn2n5msrmnx3p3ma6e6pnwxdg55edxkea4rr552wvsggewjka"},"v":{"$link":"bafyreidsgz2iggc6cme33ifqv6qhqryckzqgug6m54vwk7ujm2opqdcsrm"}},{"k":{"$bytes":"dW11b2wycA"},"p":28,"t":{"$link":"bafyreiatsrovoo2bsdboya6ke2vwbjukwl6yjplwjsza73f3spzj3cla7u"},"v":{"$link":"bafyreibf6gwd7jwnmzj6fkqgag2lo44b3bt3yas472ipg6o75ptadz6aki"}},{"k":{"$bytes":"ejNmYTMyeg"},"p":28,"t":{"$link":"bafyreihf4w6eli2rlv6ech5qkuqhjbhmgk346rx3ylokajo4myooc5saga"},"v":{"$link":"bafyreifpgldsbm73sqfdp5i5gynkbbgfkqw3pvg7mfak4ep44mu4lmdue4"}},{"k":{"$bytes":"N3RtaXA1MjI"},"p":27,"t":{"$link":"bafyreifstbh7g5cpicgztdsbminxwyu5h2u5h6qwtj4cpjw5aml5f53quy"},"v":{"$link":"bafyreigm5yanqx3wkpzmir6nmcf3pwrswvnngfwqlfi4t2g35fjiynrbty"}},{"k":{"$bytes":"b2poN3plMnk"},"p":27,"t":null,"v":{"$link":"bafyreih4geco7zdubiq344xubbd5fihzmw3fuzz2nxw6oks7sqecfa37vy"}},{"k":{"$bytes":"djJ4"},"p":32,"t":{"$link":"bafyreiawqhw5tr2dhj3f5pbrujxw7y3skmcdn33htffth4qzchwxqmcmpq"},"v":{"$link":"bafyreic6zbdxo54cwbot6rembirjzg547amizk3tosuuv4pjbatczvf4zy"}},{"k":{"$bytes":"aTVjazJ2"},"p":29,"t":null,"v":{"$link":"bafyreif3l7rk46bjx6gg2bm5q6tvygj2p7htpc4ichbfuiixjaghg7rrby"}},{"k":{"$bytes":"YzZ0Mnk"},"p":30,"t":{"$link":"bafyreihdfzm6vleayuoozlrva5k764jskfur6zsncnkg32csh5kv6k2g3a"},"v":{"$link":"bafyreib2gu6uav7loil4far7bhlffhqn2rtxougvdgtsw5jgkpukceiazu"}},{"k":{"$bytes":"a3VlZHkyNA"},"p":28,"t":{"$link":"bafyreiahvbm53bhdbyh6sgcc2qdxfbywukblac5ac4iukjlb7jpgulseum"},"v":{"$link":"bafyreifwz24gdkkzrjjeofvmvzqiudresczctod5ebrdz6ct7kyn2r63uq"}},{"k":{"$bytes":"cjRneTM0NDJ3"},"p":26,"t":null,"v":{"$link":"bafyreichw3xad2wjioulk24dui46bxtscym4rydx3za3v4uzexgwaceq4a"}},{"k":{"$bytes":"NzVsdWVsMmU"},"p":27,"t":{"$link":"bafyreiaptz4nvfuzlazrfldmlbkjcvjt5y4r3zecv5jyqlm3uvp77dcqtq"},"v":{"$link":"bafyreibjwz4qp5t7z4gx33mwo3n6nqtcenlnpvmv4yaurwrc2z6tvmxwca"}}],"l":{"$link":"bafyreibtiicxjinis5d5t4axniluwhvjhvnorxvuzkpjac7bduzlwsoqmi"}}

{"e":[{"k":{"$bytes":"YXBwLmJza3kuZmVlZC5wb3N0LzNrZ2F5bm5kczd2Mjc"},"p":0,"t":null,"v":{"$link":"bafyreifyg5hgu2wtt5gju45dhy6lsaugcoceu6dgxaxhvzmhiigfpw2hmq"}},{"k":{"$bytes":"ZXJoejJs"},"p":26,"t":null,"v":{"$link":"bafyreias6fw27tygaauei4uo2k32laphrrrriihxvjfc5ur3hsrogs24b4"}},{"k":{"$bytes":"eGRnMnY"},"p":27,"t":null,"v":{"$link":"bafyreidxjaojabwiysm3i2qsx2vkoh6sokjb532mtn55nciartpil6g2zy"}},{"k":{"$bytes":"ZndtNzJ4"},"p":26,"t":null,"v":{"$link":"bafyreihoim7qe4bbuobtgluymbul4nwbqvc5u3x7e44y6uwd4zp77rnium"}},{"k":{"$bytes":"Z2ZhbjIy"},"p":26,"t":null,"v":{"$link":"bafyreibldaiqpnms72c4eqklc5f3gdm3ucitb4djh72mand4euhf27z2hu"}},{"k":{"$bytes":"cmVwb3N0LzNrZnR2Mm8yaGJzMnc"},"p":14,"t":null,"v":{"$link":"bafyreigd5dyr5wqcqnbfxhoxp3ovl5tz4bhjwxapgf447lnkf2f4skwute"}},{"k":{"$bytes":"Z3JhcGguYmxvY2svM2tnM3U3aGFjazQydg"},"p":9,"t":null,"v":{"$link":"bafyreiem75k3qbzjuxvgxkucqpolodepn62uubs2xp6ocl4qacnl3hf5ay"}}],"l":null}

There does not seem to be any section there that would contain the record data.

In the firehose from the account's PDS (morel), the same event seems to be emitted correctly, with record data included (event 13785285):

{"text"=>"", "$type"=>"app.bsky.feed.post", "embed"=>{"$type"=>"app.bsky.embed.images", "images"=>[{"alt"=>"", "image"=>{"ref"=>{"$link"=>CID("bafkreigbbxfwwod4qfrliapj7rbrjicq2bqm5dxcwe2max3strzlvgbune")}, "size"=>736891, "$type"=>"blob", "mimeType"=>"image/jpeg"}, "aspectRatio"=>{"width"=>2000, "height"=>1500}}]}, "langs"=>["en"], "createdAt"=>"1941-12-06T03:35:47.396Z"}

^ I've just noticed that the record has a createdAt date in 1941, which might be the source of the problem?

[mackuba]

Ah, turns out I've actually written down a detailed log of that previous event too 😄 https://gist.github.com/mackuba/612c89139cbeeb74eff74e427ae38301

[whyrusleeping]

definitely seeing some wonky logs on the relay from that DID. I wonder if something is failing when it tries to fetch the fresh repo?

[whyrusleeping]

why@sirius ~/c/indigo (main)> ./gosky debug inspect-event --host="wss://bsky.network" 434007890
{
  "blobs": null,
  "blocks": {
    "$bytes": "OqJlcm9vdHOB2CpYJQABcRIgEQilOZXbItKH42voWzvkUWzo98fGrnp5uSgIb5fVUUBndmVyc2lvbgHRAQFxEiCtL2SvyehGzMwi1ltnX/5KqrZWYcvYzHm/U9pHEoXDiKJhZYGkYWtYIGFwcC5ic2t5LmZlZWQucG9zdC8za2dheW5vdmkyejJsYXAAYXTYKlglAAFxEiC1b149ghDDDT5+DgKtRnP+aXZbmwPGv7+N31cym6dxsGF22CpYJQABcRIg4MXYIg+AwsVAHLMaQsq418VbAEDY01Ssh6dRqYrc12xhbNgqWCUAAXESICbJUTmxD0P8qkd5HikKg2B6wGnwKEehH3fGqTxc5Yc5UwFxEiC9rSIgK38bmbRFs2joqkzpqHrgGhRTlTwzKfSjBNVlyKJhZYBhbNgqWCUAAXESIGZdjYChuDX14erlHSZ9gmWyFe7sJKtUIl7abI0VG8jZuBABcRIgZl2NgKG4NfXh6uUdJn2CZbIV7uwkq1QiXtpsjRUbyNmiYWWVpGFrWCBhcHAuYnNreS5mZWVkLnBvc3QvM2tnYXluczM0eXcyeWFwAGF02CpYJQABcRIgd2BBaYkh5ZmfnUummGLPGyzROG/grA93ffC5XZ21xrBhdtgqWCUAAXESIE6NbLuQ8+q2jFPp+PgFCAXUSOk4YGsAt5FZW5VLryQdpGFrRjdnb3cyYWFwGBphdNgqWCUAAXESIHlkfcgLH5BJGuGCAe2Z+xNtHAGYkvI8YRdItv4K62HqYXbYKlglAAFxEiC3hU7p/6GraQ1I7a1YJEgX6ml2tMgB634FvQItuvvCOKRha1gZZ3JhcGguYmxvY2svM2tnM3U3dWxnajUybWFwCWF02CpYJQABcRIgsC3+7yuyGzMGRaqTaNDSSZj4iM5+YvmTRNeySltoVgNhdtgqWCUAAXESIHsVB9Y4H3XvP7lbx2dHCgteDlO5YhHy4gp0maojh64EpGFrSGI3cXFudTI3YXAYGmF02CpYJQABcRIgDx+AMycylgHk9x1IcR/zsAN4mqJ5wBQoyCWzshQ+4wVhdtgqWCUAAXESIHxdcRVnWyySowKoy90233ekyf5YQQukrBIx4njuVG2fpGFrSGhvaHg2azJsYXAYGmF02CpYJQABcRIg8TUKnR+WqSnxc94BgIeeexh39PHybtWKdd9oNvFPhcJhdtgqWCUAAXESICz+4DW1Gtav9aEf0Jx9EeiKyTmQJFbP1foJqt2MB+Z2pGFrSGl4YnFkejJ1YXAYGmF02CpYJQABcRIgXCgo4y5zGiWI+K6iqeTNG6TDRr/B7p8Lb2WQR4tijZ5hdtgqWCUAAXESIOhEZLEUhRNb5WxUsOwsQ0At0U4tkQ/gIihttQWOHdaepGFrSG5ibDVuNTIyYXAYGmF09mF22CpYJQABcRIgkFcjGkQq6UOOI/IApxpOaN567MKRqrZL69j5e3gsuiCkYWtIeWhna3d2MjNhcBgaYXTYKlglAAFxEiB9pqlrUOGGcFH5YFLGn/clnTTA5OhkF8X5PGXjjZdHNGF22CpYJQABcRIgOiWY3AO62LVU3gGPDQZWDKSxbMudsOH3znIAsYC9K8ikYWtUZm9sbG93LzNrZnhuenc0NzJ0MmthcA9hdNgqWCUAAXESIAsOx1bK5dk5tZxqd7lJSB7tzf01Le5EU6Qk3XjI9WtcYXbYKlglAAFxEiCw385isXb/0laVJfsq3CoQ0GKTE4xi+qXGnmL6cwd+eaRha0lvNDU3dmRjMmxhcBgaYXTYKlglAAFxEiAQ6Ydk9Yt+J7y6QjioUu0q9UDWk8FdrEjKmi92w3e6U2F22CpYJQABcRIgjryVyotILKewZWWj2SAuL2k0xMiz3okwTbNBn4vV5l6kYWtINXI3NHhsMnZhcBgbYXTYKlglAAFxEiCnXNum9ZKLG32/bAeJ57bXGbvSDuogORj3urVkYxLJUGF22CpYJQABcRIgcjZ0gxheEwm9oLCvoHhHAlZgahvM7ytlfolmnPgMUoukYWtHdW11b2wycGFwGBxhdNgqWCUAAXESIBOUXVc7QZDC7APKJqtgpoqy/YS9dkyyD+y7k/KdiWD9YXbYKlglAAFxEiAl8aw/ps1mU+KqBgG0t3OB2Ge8Alz+kPN53+vmAefAUqRha0d6M2ZhMzJ6YXAYHGF02CpYJQABcRIg5eW8RaNRXXxBH7BVIHSE7DK3z0b7wtygJdxmHOF2QDBhdtgqWCUAAXESIK8yxyCz+5QKN/UdNhqghMVULbfU32FArhH84ynFsHQnpGFrSDd0bWlwNTIyYXAYG2F02CpYJQABcRIgsphP83RPQI2ZjkFiG3tinT6p0/oWmngnpt0DF9L3cKZhdtgqWCUAAXESIMzuANhfdlPyxEfNYIu32jK1WtMW0FlRyejb6VKMNiGepGFrSG9qaDd6ZTJ5YXAYG2F09mF22CpYJQABcRIg/DEE7+R0CiG+cvQIR9Kg+WW2Wmc6be3nKl+UCCKDf66kYWtDdjJ4YXAYIGF02CpYJQABcRIgFoHt2cdDOnZevDGib2/jclMENu9nmUsz8hkR7XgwTHxhdtgqWCUAAXESIF7IR3d3grBdP0SMCiKcm7z4GIyrc3SpSvHpCCYs1LzOpGFrRmk1Y2sydmFwGB1hdPZhdtgqWCUAAXESILtf4q54Kb+MbQWdh6dcGTp/zzeLiBHCWiEXSAxzfjEOpGFrRWM2dDJ5YXAYHmF02CpYJQABcRIg4y5Z6qyAxRzsrjUHVf9xMlFpH2ZNE1Rt6FI/VV8rRthhdtgqWCUAAXESIDo1PUBX63IXwoI/CdZSng3UZ3dQ1RmnK3UmU+ihEQDNpGFrR2t1ZWR5MjRhcBgcYXTYKlglAAFxEiAHqFndhOMOD+kYQtQHcocWooKwC6AXEUUlYfpeai5Eo2F22CpYJQABcRIgts64YalZilJHFqyuYIoOJJCyKbh9IGI8+FP6sN1H26SkYWtJcjRneTM0NDJ3YXAYGmF09mF22CpYJQABcRIgR7buAerJQ6i1a4OiOeDechYZyOB33kG68pklzWAIkOCkYWtINzVsdWVsMmVhcBgbYXTYKlglAAFxEiAPnnjalplYMxKsbFhUkVUz7jkd5IKvU4gtm6Vf/4xQnGF22CpYJQABcRIgKbZ5B/Z/zw197ZZ22+bCYiNW19WV5gFI2iLWfTqy9hBhbNgqWCUAAXESILFoMlvspR75EqEYDhNZsogTqUtEbDhxyuxW1Et+LDSU4AEBcRIgEQilOZXbItKH42voWzvkUWzo98fGrnp5uSgIb5fVUUCmY2RpZHggZGlkOnBsYzpoNTJpdXA1ZXd2cGttMm9hZWlwZG14Z3RjcmV2bTNrZ2F5bnM3dmRvMmFjc2lnWEBe7FvU0gUaWivPl7N5S5YQmoVi6hosap2+D+kC7d0/7Ta7yLZYlhMXigo+RRqd8taQCHxYhpO6eZ9E/72zUsr7ZGRhdGHYKlglAAFxEiAfg6i2OkKtQWMsup9ZeGv8tsSvpVA3LougK0kSv+LWwWRwcmV29md2ZXJzaW9uA4YEAXESIB+DqLY6Qq1BYyy6n1l4a/y2xK+lUDcui6ArSRK/4tbBomFlhKRha1ggYXBwLmJza3kuZmVlZC5wb3N0LzNrZnVwem5zc2R0MmFhcABhdNgqWCUAAXESILHbmYYrYffAvksV7650Jgf67YPtIrAm65nzS1UtZn99YXbYKlglAAFxEiCPATcqRTsAURhMOjtbXkvmi9gA5zKyeHmHWq0vlPD1wqRha0l6am94NzM0MmZhcBdhdNgqWCUAAXESINFCQyGNA3QdDRlBv50DcovCVUlyiWYkk45j1W9hLhAbYXbYKlglAAFxEiBJ57rhO3Cu6EC0IPtV0mqNQmYkrI562F0FsM5BjNjsu6Rha0p5Y3pqb3AyMzJwYXAWYXTYKlglAAFxEiDLzFhTXGC3fRUEZ7Ji9pqBbOMVcBRCHx+uvzgRzvt6AWF22CpYJQABcRIghggWyZk41QVozyI/XAncKKBDI221Fvy2h90m5197EKmkYWtLZ2F5bmV6ZXRkMnNhcBVhdNgqWCUAAXESIL1LiKJbRJ0zF8dStCP8w/fow9Eh20EsrL9FP4HiM53gYXbYKlglAAFxEiBAFNbFFt0dtf7GuICK/+pisN5TujkufaB63/OFVAyGQGFs2CpYJQABcRIgik0gA7IjJVLCgWEgaSfvP0jrfLSJcyiQDZWvjx6FZZLRAQFxEiC9S4iiW0SdMxfHUrQj/MP36MPRIdtBLKy/RT+B4jOd4KJhZYGkYWtYIGFwcC5ic2t5LmZlZWQucG9zdC8za2dheW5sdmJ4NjJlYXAAYXTYKlglAAFxEiB3ZSdHSMI3wdEOdrXaKxJ/x33n9x+5IzSiWk6VTPDRqWF22CpYJQABcRIgzuTU/VJPljHIOpJKx5C3rkkgN57I18oKQhmbLJ1T5ldhbNgqWCUAAXESIHnLmj1wo2U2ssStM5/BaJMedMbvEtZXhaJ1mDsn+W+LUwFxEiB3ZSdHSMI3wdEOdrXaKxJ/x33n9x+5IzSiWk6VTPDRqaJhZYBhbNgqWCUAAXESIK0vZK/J6EbMzCLWW2df/kqqtlZhy9jMeb9T2kcShcOItgIBcRIgtW9ePYIQww0+fg4CrUZz/ml2W5sDxr+/jd9XMpuncbCiYWWCpGFrWCBhcHAuYnNreS5mZWVkLnBvc3QvM2tnYXlucmdodXgyaWFwAGF02CpYJQABcRIgv87CnrMOLbsiUeowY7hU+k5pCj2Hh2OZQZSkvmmN085hdtgqWCUAAXESIBKRJFMjIsiO6n1A3IaZkeNh/t3EofHQgPYbE96Zjhx3pGFrR3MyNXFuMndhcBgZYXTYKlglAAFxEiC9rSIgK38bmbRFs2joqkzpqHrgGhRTlTwzKfSjBNVlyGF22CpYJQABcRIg55LHwNvQe6fOySVS+R3bFWvW8F5m8hQk5BdlKyQdGmVhbNgqWCUAAXESICvyFPbEKwJL0T+Qb2RlXvkm9z0t28Cr58blQqHV28ovrAMBcRIgd2BBaYkh5ZmfnUummGLPGyzROG/grA93ffC5XZ21xrCiYWWGpGFrWCBhcHAuYnNreS5mZWVkLnBvc3QvM2tnYXluczNxancyZmFwAGF09mF22CpYJQABcRIgQXFlxhRGdN29t225Vywl8MNUj6yp9JN9XowWEADT/bekYWtGNHRvbzJyYXAYGmF09mF22CpYJQABcRIgHnlmWK8A211h4uEtOkS0D+7saI90mjcqwWCSflpxrmmkYWtGNWJlazJiYXAYGmF09mF22CpYJQABcRIgfvIq9WuTiEK7QhLbyZ0gTGz5zJnSfc7q3sGii+sTAZWkYWtFdXY1MmxhcBgbYXT2YXbYKlglAAFxEiD0iqutSkn/Q4wYiZIbMKKjUbF6rFgwnFGaS+/qsWSPV6Rha0Y2bGU2MmVhcBgaYXT2YXbYKlglAAFxEiDgkyCh3hVliLCsZi0Bb1pDvC0z7TYMZG+qU5ednIJ/tqRha0Y3Y3J1MmthcBgaYXT2YXbYKlglAAFxEiCponE8SI6skC/M4t0SH+BW0KJKyY4Cw9QthD74J2dIR2Fs9tIBAXESIHlkfcgLH5BJGuGCAe2Z+xNtHAGYkvI8YRdItv4K62HqomFlgqRha1giYXBwLmJza3kuZmVlZC5yZXBvc3QvM2tmdHYybzJoYnMyd2FwAGF09mF22CpYJQABcRIgw+jxHtoCg0Jbndd+3VX2eeBOm1wPMXnPraoui8kq1JmkYWtYGWdyYXBoLmJsb2NrLzNrZzN1N2hhY2s0MnZhcAlhdPZhdtgqWCUAAXESIIz/VbgHKaXqa6qCg9y3DI9vtUoGWrv84S+QAJq9nL0GYWz2"
  },
  "commit": {
    "$link": "bafyreiarbcsttfo3eljipy3l5bntxzcrntuppr6gvz5htojibbxzpvkria"
  },
  "ops": [
    {
      "action": "create",
      "cid": {
        "$link": "bafyreia6pftfrlya3nowdyxbfu5ejnap53wgrd3uti3svqlasj7fu4none"
      },
      "path": "app.bsky.feed.post/3kgayns4too2r"
    },
    {
      "action": "create",
      "cid": {
        "$link": "bafyreid66ivpk24trbblwqqs3pez2icmnt44zgospxhovxwbukf6weybsu"
      },
      "path": "app.bsky.feed.post/3kgayns5bek2b"
    },
    {
      "action": "create",
      "cid": {
        "$link": "bafyreihurkv22ssj75byygejsintbivdkgyxvlcygcofdgsl57vlczepk4"
      },
      "path": "app.bsky.feed.post/3kgayns5uv52l"
    },
    {
      "action": "create",
      "cid": {
        "$link": "bafyreihasmqkdxqvmwelbldgfuaw6wsdxqwth3jwbrsg7ksts6ozzat7wy"
      },
      "path": "app.bsky.feed.post/3kgayns6le62e"
    },
    {
      "action": "create",
      "cid": {
        "$link": "bafyreifjujytyseovsic7thc3ujb7ycw2crevsmoalb5ilmeh34coz2ii4"
      },
      "path": "app.bsky.feed.post/3kgayns7cru2k"
    },
    {
      "action": "create",
      "cid": {
        "$link": "bafyreifxqvhot75bvnuq2shnvvmcisax5juxnngiahvx4bn5aiw3v66cha"
      },
      "path": "app.bsky.feed.post/3kgayns7gow2a"
    }
  ],
  "prev": null,
  "rebase": false,
  "repo": "did:plc:h52iup5ewvpkm2oaeipdmxgt",
  "rev": "3kgayns7vdo2a",
  "seq": 434007890,
  "since": "3kgayns3vg62f",
  "time": "2023-12-11T08:23:53.406Z",
  "tooBig": false
}

Slice Dump:
Root:  bafyreiarbcsttfo3eljipy3l5bntxzcrntuppr6gvz5htojibbxzpvkria
bafyreifnf5sk7spii3gmyiwwlntv77skvk3fmyol3dghtp2t3jdrfbodra
bafyreif5vurcak37dom3irntndukuthjvb5oagqukoktymzj6srqjvlfza
bafyreidglwgybinygx26d2xfduth3atfwik653bevnkcexw2nsgrkg6i3e
bafyreiarbcsttfo3eljipy3l5bntxzcrntuppr6gvz5htojibbxzpvkria
bafyreia7qoulmoscvvawglf2t5mxq274w3ck7jkqg4xixibljejl7ywwye
bafyreif5joekew2etuzrpr2swqr7zq7x5db5cio3iewkzp2fh6a6em454a
bafyreidxmutuosgcg7a5cdtwwxncwet7y566p5y7xertjis2j2kuz4grve
bafyreifvn5pd3aqqymgt47qoakwum476nf3fxgydy2737do7k4zjxj3rwa
bafyreidxmbawtcjb4wmz7hklu2mgfty3ftitq37avqhxo7pqxfoz3nogwa
bafyreidzmr64qcy7sbervymcahwzt6ytnuoadges6i6gcf2iw37av23b5i

Ops:
loading "app.bsky.feed.post/3kgayns4too2r": ipld: could not find bafyreia6pftfrlya3nowdyxbfu5ejnap53wgrd3uti3svqlasj7fu4none

inspecting one of the events from the relay shows the missing block.

Looking closer at their repo, they have a ton of posts where the cid is the exact same as other posts in their repo (in some cases there over 40 posts with the same cid but different rkey).

[mackuba]

Check out the two from that older gist too (one is an "Elon Musk mirror" and the other looks mostly normal).

[bnewbold]

Is this still an open issue? I vaguely remember that we started including blocks in events coming out of Relay even if they had been included in the MST earlier, to make things easier on downstream consumers. Did that resolve this issue?