search

snarfed / bridgy-fed

🌉 A bridge between decentralized social network protocols
https://fed.brid.gy/
Creative Commons Zero v1.0 Universal
486 stars 28 forks source link

Double post in replies fediverse -> AtProto #1063

Open TomCasavant opened 1 month ago

TomCasavant commented 1 month ago

https://bsky.app/profile/tom.tomkahe.com.ap.brid.gy/post/3ksumarmd4j52 top level post https://bsky.app/profile/tom.tomkahe.com.ap.brid.gy/post/3ksuuv77l5pn2 reply https://bsky.app/profile/tom.tomkahe.com.ap.brid.gy/post/3ksuuv6ivb2p2 duplicate reply (https://tomkahe.com/@tom/112469674016938064)

I think this happened to me a few weeks ago as well but I can't find it. In both cases it was just in the replies and not a top level post

snarfed commented 1 month ago

So odd! Sorry for the trouble. I got another report of this today too, https://github.com/snarfed/bridgy-fed/issues/947#issuecomment-2119758062, but that one was seemingly caused by editing the Mastodon reply, which doesn't look like it happened here. No clue what's going on yet.

TomCasavant commented 1 month ago

No worries, just figured it's one of those weird bugs you'd want examples of to diagnose

TomCasavant commented 1 month ago

Oh that's interesting, same thing popped up when I replied to your post. https://snarfed.org/2024-05-20_53092

image

snarfed commented 1 month ago

Yes! I saw that too. Those are webmentions, but maybe still related. And ideally BF wouldn't send two, but they have the same source URL, and webmentions should be idempotent, so the fact that two separate comments got created is likely a bug in the WordPress webmention plugin.

snarfed commented 1 month ago

Aha. One part of the root cause here is that we're getting the same activity delivered to us multiple times to different hosts. Eg @TomCasavant's instance delivered the reply activity in https://github.com/snarfed/bridgy-fed/issues/1063#issuecomment-2121150166 twice, once to web.brid.gy and once to bsky.brid.gy.

685 would help here, but I'd also like to make sure that processing is idempotent all the way through regardless. It is for sending webmentions, but evidently not for creating records in ATProto repos, we're duplicating the same post with different tids (rkeys). Hrm. Ideally we should map input objects to tids deterministically.

snarfed commented 1 month ago

I made an infrastructure change yesterday that should hopefully prevent these duplicates. Please let me know if you see any new ones!

I still want to map AP ids to ATP tids deterministically, I'll leave this open to track that.

snarfed commented 1 month ago

@TomCasavant have you seen this happen any more in the last week? I may skip making ids => tids deterministic, the new idempotence may be enough.

TomCasavant commented 1 month ago

I haven't seen it happen again

snarfed commented 1 month ago

Thanks!

mackuba commented 1 month ago

This still seems to be happening in some form (not sure if it's the same issue)...

Example: Post: https://bsky.app/profile/did:plc:zqnkaserlfmoyfizucbtmpw4/post/3ktokhwljg4f2

If you query for did:plc:zqnkaserlfmoyfizucbtmpw4 and 3ktokhwljg4f2 in https://atproto.tools, you get two post creates and no updates or deletes: https://atproto.tools/records?did=did:plc:zqnkaserlfmoyfizucbtmpw4&collection=app.bsky.feed.post&rkey=3ktokhwljg4f2

Screen Shot 2024-06-01 at 00 55 53

They're 189 seqs apart, so that's something like 3-4 seconds, and they have the exact same JSON content. Also the original post at https://indieweb.social/@jaredwhite/112528096281782994 doesn't seem to have been edited, so it looks more like some kind of race condition thing.

A query in my database for duplicate rkeys from the Bridgy PDS shows several such cases from the last 2 days:

repo                              rkey           n
--------------------------------  -------------  -
did:plc:3zmgk6ltrvrobp7uuoechqnl  3ktrhwl7mcca2  2
did:plc:6oopxocfjkevkh3ktunkct3f  3ktqvrttltpa2  2
did:plc:aot633jq4fzdbejqflolsq4h  3ktrh27lk64a2  2
did:plc:e7swajiiqv5deci34fn2pon5  3ktrlfbexb2a2  2
did:plc:f4wptjuakccjndanesgcpbyy  3ktsn3cmguza2  2
did:plc:fw5hgb3x45xeazgtmphhuyx7  3ktoh3swd5bf2  2
did:plc:gg77efagxhons7wwxbjfeqrx  3ktqrt7yq2ta2  2
did:plc:gg77efagxhons7wwxbjfeqrx  3ktr4ycvkmxa2  2
did:plc:h5cyfgzmibjc7c5n5ph5czqr  3ktp7ikolnbw2  2
did:plc:haymkinjurgmo37dl4wmbtpo  3ktsjcrb4sda2  2
did:plc:hd4cgld5pxpy7kpsilxu62u6  3ktovf3qrdhw2  2
did:plc:hncngtu4hleej54nqvqy3zve  3ktrjxugvvba2  2
did:plc:ju6k7dob4f74h5rdymughwz6  3ktsn3nv7dpa2  2
did:plc:kir7hkjqs6k4zmnix6r5hbw4  3ktontigs7cw2  2
did:plc:lglhkumc3c7mecjxfdlzwyux  3ktsx6b4dced2  2
did:plc:lxf6nbzgcphkzhbjzdhz24wa  3ktqhx6yl42w2  2
did:plc:twerjlabpqmvieyx3roydios  3ktpjlfggohw2  2
did:plc:zfnr7z743mhdmnljqgmytllw  3ktooak533gw2  2
did:plc:zqnkaserlfmoyfizucbtmpw4  3ktokhwljg4f2  2
snarfed commented 1 month ago

@mackuba interesting, thanks! Yeah that's a different issue. This issue was for user-visible duplicates, ie the same fediverse post resulting in two different Bluesky posts with different tids.

Multiple create events for the same tid/record obviously isn't ideal either, you're right, but definitely lower priority.

TomCasavant commented 3 days ago

I've got another case of this:

https://bsky.app/profile/emarktaylor.mastodon.online.ap.brid.gy/post/3kw3spmzwxpf2

https://bsky.app/profile/emarktaylor.mastodon.online.ap.brid.gy/post/3kw3spmw7fqi2

https://mastodon.online/@emarktaylor/112702065045640052

snarfed commented 3 days ago

Ugh. Thank you! Reopening. Seems pretty clear there's a race condition here somewhere that I need to track down.