Facebook comments linked to wrong post

[beck24]

Comments from this post (FB: https://www.facebook.com/photo.php?fbid=1080330665330686 original: http://known.mattbeckett.me/2015/day-17-devilish-demands)

Are showing up on this post: http://known.mattbeckett.me/2015/new-plugin-elgg-copy

I don't think it's the same issue as before (where the photos are grouped) as the elgg_copy plugin post isn't an image post and is from 23 days ago.

[beck24]

These last few pictures did get grouped into one of those statuses, but it doesn't include the plugin post https://www.facebook.com/matt.beckett.39/timeline/story?ut=43&wstart=0&wend=1451635199&hash=-4645984981838288132&pagefilter=3

(doesn't appear to be available while logged out despite all photos being public, shoot me a friend request if it helps)

Actually it looks like some (but not all comments) from each of the grouped pictures ended up on the plugin post.

[snarfed]

ugh, thanks for letting me know. and sorry this keeps happening. :/

looking at https://brid.gy/facebook/948590525171368 , it looks like we started sending them to the right post ~5h ago. we cache the mapping between facebook post objects and photo objects, so i suspect we had an old mapping cached. still badly wrong, given that the elgg post isn't a photo at all, so that part is still confusing. definitely worth investigating more!

[snarfed]

hey @beck24, have you seen this happen on any more posts recently?

[beck24]

Last occurrence I see is from a week ago: http://known.mattbeckett.me/2015/day-21-update-2#comments

Those comments belong to this post: http://known.mattbeckett.me/2015/day-21-scene-se7en

Since then it seems comments haven't been coming through, or are inconsistent...

http://known.mattbeckett.me/2015/day-22-baited-baddies#comments

http://known.mattbeckett.me/2015/day-22-update---mystery-solved#comments

[beck24]

@snarfed sorry to bring you some bad news...

The comments here: http://known.mattbeckett.me/2016/john-scott-on-his-road-to-the-all-star-game#comments

Belong here: http://known.mattbeckett.me/2016/a-new-level-of-stupid#comments

[snarfed]

damn. sorry! bad news indeed. thanks for letting us know.

[beck24]

ok, this is getting confusing.

http://known.mattbeckett.me/2016/i-have-named-my-website-many-fine-books-so-now#comments

Has comments from this post:

They have nothing in common, were posted over 24 hrs apart, only the one post is syndicated. The one about my kids is also set to privacy:friends so those comments shouldn't have been pushed anywhere.

[snarfed]

I'm so sorry this is still happening to you, Matt, and especially with non-public posts and comments! that's definitely not OK, and obviously not what we want.

i haven't made much more progress on finding the root cause yet, nor have i found anyone else who's seeing the same bug. I'll definitely keep looking, though. wish me luck.

[beck24]

Yeah I don't know. I guess it's good in a way that I'm the only one, but I'm not sure why I'm the only one. I don't think I'm doing anything out of the ordinary with it, it's pretty much a stock known installation with a couple of custom display/analytics plugins, but nothing that even touches syndication. I do hope you can find something though, good luck :)

[snarfed]

links:

• source and auth datastore entities
• recent Response datastore entities
• original post on known.mattbeckett.me
• FB public post (1109803145716771)
• FB private post (1109129219117497)
• FB comment on private post (1109286935768392)
• bridgy handler for FB private comment (400s)
• SyndicatedPost datastore entity. syndication correctly points to the public post, and created == updated, so it doesn't look like it changed at all.
• propagate log
• poll log

• FB public post in API explorer:

  {
  "id": "948590525171368_1109803145716771",
  "type": "status",
  "created_time": "2016-02-06T06:20:10+0000",
  "updated_time": "2016-02-06T17:25:38+0000",
  "message": "I have named my website \"many fine books\". So now I can be referenced as \"Matt Beckett - developer, scuba diver, and author of many fine books\".
  
  (known.mattbeckett.me/s/3Vl2R)",
  "comments": {"data": ["..."]},
  "from": {
    "name": "Matt Beckett",
    "id": "948590525171368"
  },
  "is_hidden": false,
  "is_expired": false,
  "likes": {"data": ["..."]},
  "privacy": {
    "allow": "",
    "deny": "",
    "description": "Public",
    "friends": "",
    "value": "EVERYONE"
  },
  "..."
  }

• FB private post in API explorer:

  {
  "id": "948590525171368_1109129219117497",
  "type": "status",
  "status_type": "mobile_status_update",
  "created_time": "2016-02-05T00:29:11+0000",
  "updated_time": "2016-02-05T12:22:50+0000",
  "message": "...",
  "comments": {"data": ["..."]},
  "from": {
    "name": "Matt Beckett",
    "id": "948590525171368"
  },
  "is_hidden": false,
  "is_expired": false,
  "likes": {"data": ["..."]},
  "privacy": {
    "allow": "",
    "deny": "",
    "description": "Your friends",
    "friends": "",
    "value": "ALL_FRIENDS"
  },
  "..."
  }

• FB private comment in API explorer:

  {
  "id": "1109129219117497_1109286935768392",
  "created_time": "2016-02-05T06:34:11+0000",
  "from": {
    "name": "...",
    "id": "1028054227212961"
  },
  "message": "...",
  "can_remove": false,
  "like_count": 2,
  "user_likes": true,
  }

• private comment Response.response_json:

  {
  "objectType": "comment",
  "id": "tag:facebook.com,2013:1109129219117497_1109286935768392",
  "published": "2016-02-05T06:34:11+00:00",
  "content": "...",
  "fb_id": "1109129219117497_1109286935768392",
  "url": "https:\/\/www.facebook.com\/1109129219117497?comment_id=1109286935768392",
  "author": {
    "objectType": "person",
    "id": "tag:facebook.com,2013:1028054227212961",
    "image": {
      "url": "https:\/\/graph.facebook.com\/v2.2\/1028054227212961\/picture?type=large"
    },
    "url": "https:\/\/www.facebook.com\/1028054227212961",
    "displayName": "...",
    "numeric_id": "1028054227212961"
  },
  "inReplyTo": [
    {
      "id": "tag:facebook.com,2013:1109129219117497",
      "url": "https:\/\/www.facebook.com\/1109129219117497"
    }
  ]
  }

• ...and Response.activities_json (it's the public post):

  [{
  "id": "tag:facebook.com,2013:1109803145716771",
  "fb_id": "948590525171368_1109803145716771",
  "object": {
    "content": "I have named my website 'many fine books'. So now I can be referenced as 'Matt Beckett - developer, scuba diver, and author of many fine books'.\n\n(known.mattbeckett.me\/s\/3Vl2R)"
  },
  "url": "https:\/\/www.facebook.com\/948590525171368\/posts\/1109803145716771"
  }]

[snarfed]

notable: both the public and private FB posts are simple status updates. neither has photo(s) or a link. so this probably isn't our post <=> photo object mapping after all.

[snarfed]

we currently check that posts and respones are public in poll and propagate, but not in the mf2 handlers. checking there too would be a good safeguard to add. it wouldn't fix the root cause bug, but it would at least prevent backfeeding non-public responses.

...the catch is that FB doesn't actually include the privacy field in many individual response objects, eg comments. you have to fetch the parent post/object and use its value instead. we already do this for photos/albums, https://github.com/snarfed/bridgy/issues/559#issuecomment-159642227, so it might still be worthwhile.

[snarfed]

btw, for the record, i do expect this bug is hitting other people too. i just haven't heard from or found any of them yet.

[snarfed]

interesting. the Response key name tag:facebook.com,2013:1109129219117497_1109286935768392 uses the private post's id, but Response.activities_json is the public post.

[snarfed]

phew, i think i found it. we make multiple FB API calls to get all of the different kinds of posts and responses, and then do some bookkeeping to match responses with their posts. as our FB API code grew and evolved, that bookkeeping didn't entirely keep up, so in some cases it was attaching responses to the wrong posts.

thanks again for your patience and persistence, @beck24! this was an ugly bug, so i'm glad it's (hopefully!) fixed. i'll keep this issue open for a while until we're confident it's gone.

[beck24]

Awesome, thanks for digging deep on it. I'll keep you posted if anything else comes up, but I'm looking forward to things just ending up in the right places :)