aaronpk / webmention.io

Easily enable webmentions on any web site
https://webmention.io/
Other
774 stars 38 forks source link

Some Bridgy Fed wms failing with `no_link_found` #187

Closed snarfed closed 1 year ago

snarfed commented 1 year ago

webmention.io is oddly rejecting some Bridgy Fed webmentions with no_link_found, even though the source page does link to the target URL. Originally filed in https://github.com/snarfed/bridgy-fed/issues/440 by @jamietanna. One example:

  <a class="u-in-reply-to" href="https://www.jvt.me/mf2/2023/03/shs0i/"></a>

@aaronpk my only guess is the redirect, but that should be pretty common, I'm sure wm.io supports source URL redirects, right?

snarfed commented 1 year ago

This turned out to be redirects, webmention.io is blocklisting Bridgy source URL redirects but shouldn't be. Fix PR is https://github.com/aaronpk/XRay/pull/120

snarfed commented 1 year ago

Woo, https://github.com/aaronpk/XRay/pull/120 fixed this! https://webmention.io/www.jvt.me/webmention/nLYk8BQjgd8mT7Uf7CqT , visible on https://www.jvt.me/mf2/2023/03/shs0i/#webmentions

joshuap commented 10 months ago

@snarfed I'm having a similar issue w/ Bridgy + Mastodon + Webmention.io:

{
  "status": "no_link_found",
  "source": "https://brid.gy/comment/mastodon/@wood@hachyderm.io/111439777418267489/111439855894516928",
  "target": "https://joshuawood.net/webmentions",
  "summary": "The Microformats at the source URL do not contain a link to the target URL. Check the source URL in a Microformats parser such as php.microformats.io"
}

It's strange because it accepted the reply that this was in response to, but it won't accept this one. I check the microformats on php.microformats.io as suggested, and it parsed the source URL just fine afaict:

http://php.microformats.io/?url=https%3A%2F%2Fbrid.gy%2Fcomment%2Fmastodon%2F%40wood%40hachyderm.io%2F111439777418267489%2F111439855894516928

snarfed commented 10 months ago

Ugh. Sorry for the trouble, @joshuap! Not entirely sure what's going on there, but I suspect the problem is we're finding that post at least two different ways, as a downstream reply and as an @-mention of you, and we're not merging the two versions. Eg we sent this webmention for the reply, but when webmention.io fetches the source page, we render it as the @-mention, so it doesn't have u-in-reply-to pointing to https://joshuawood.net/webmentions .

I thought we fixed this a long time ago, in https://github.com/snarfed/bridgy/issues/533 and snarfed/bridgy@a10a5a1c37c6419717d940a6dee7401462975ac9, but maybe not, or maybe not all variations. Will look. I suspect this one is tricky though, no promises.

joshuap commented 10 months ago

Ugh. Sorry for the trouble, @joshuap! Not entirely sure what's going on there, but I suspect the problem is we're finding that post at least two different ways, as a downstream reply and as an @-mention of you, and we're not merging the two versions. Eg we sent this webmention for the reply, but when webmention.io fetches the source page, we render it as the @-mention, so it doesn't have u-in-reply-to pointing to https://joshuawood.net/webmentions .

I thought we fixed this a long time ago, in snarfed/bridgy#533 and snarfed/bridgy@a10a5a1, but maybe not, or maybe not all variations. Will look. I suspect this one is tricky though, no promises.

Any help or guidance you can provide will be much appreciated, thanks for looking!

stefanoverna commented 5 months ago

I think I'm having the same issue here too (Bridgy + Mastodon + Webmention.io):

{
  "status": "no_link_found",
  "source": "https://brid.gy/comment/mastodon/@steffoz@mastodon.social/112167993661516620/112167993661516620",
  "target": "https://squeaki.sh/p/being-part-of-the-indieweb",
  "summary": "The Microformats at the source URL do not contain a link to the target URL. Check the source URL in a Microformats parser such as php.microformats.io"
}

Log: https://brid.gy/log?module=background&start_time=1711549235&key=agdicmlkLWd5cjkLEghSZXNwb25zZSIrdGFnOm1hc3RvZG9uLnNvY2lhbCwyMDEzOjExMjE2Nzk5MzY2MTUxNjYyMAw

The post in question is a reply to a reply: https://mastodon.social/@steffoz/112167993661516620


But with this other reply to reply, everything worked just fine: https://mastodon.social/@steffoz/112167872530600615

Log: https://brid.gy/log?module=background&start_time=1711548983&key=agdicmlkLWd5cjkLEghSZXNwb25zZSIrdGFnOm1hc3RvZG9uLnNvY2lhbCwyMDEzOjExMjE2Nzg3MjUzMDYwMDYxNQw

snarfed commented 5 months ago

Huh, interesting. Looking at the source URL for that webmention, it has:

<a class="u-in-reply-to" href="https://mastodon.social/web/statuses/112167934611297209"></a>

...which is some kind of interstitial redirect prompt thing that mastodon.social is doing. The actual in-reply-to should be https://social.lol/@amerpie/112167934583687660 , which Bridgy would connect with your original post https://squeaki.sh/p/being-part-of-the-indieweb . I wonder if mastodon.social is replacing the social.lol URL with its prompt URL, even in data returned via the API? Hrm.

stefanoverna commented 5 months ago

Mh, ok, so it's something related to replies to-reply towards users residing in third-party servers? If you give me some hints I can try to give you an answer to that last question :) What should be the API call to try on mastodon.social?

snarfed commented 5 months ago

Sure! It's /api/v1/statuses/[id]/context. Here's how we use it:

https://github.com/snarfed/granary/blob/66ee3544cbf2852d3f03ed26c1407778ec341e36/granary/mastodon.py#L242-L247

You may not be able to identify the root cause here from looking at Mastodon API responses alone - I expect there's at least some interaction with granary's/Bridgy's logic - but feel free to try!

stefanoverna commented 5 months ago

Alright! So the response you'd like to see is of this one, right?

curl 'https://mastodon.social/api/v1/statuses/112167934611297209/context' | jq '.'

This is the result, hope it helps, and let me know if I can make your investigative work easier :)

{
  "ancestors": [
    {
      "id": "112167724013861636",
      "created_at": "2024-03-27T12:47:20.351Z",
      "in_reply_to_id": null,
      "in_reply_to_account_id": null,
      "sensitive": false,
      "spoiler_text": "",
      "visibility": "public",
      "language": "en",
      "uri": "https://mastodon.social/users/steffoz/statuses/112167724013861636",
      "url": "https://mastodon.social/@steffoz/112167724013861636",
      "replies_count": 3,
      "reblogs_count": 7,
      "favourites_count": 14,
      "edited_at": null,
      "content": "<p>It&#39;s official! I&#39;m now a proud participant in the IndieWeb! :pensive_party_blob: </p><p>Microformats, webmentions, feed, automatic backfeed from Mastodon... the whole package!</p><p>I just wished for clearer documentation.. figuring out how to interconnect the various pieces was quite a nightmare, but very simple once understood.</p><p><a href=\"https://mastodon.social/tags/indieweb\" class=\"mention hashtag\" rel=\"tag\">#<span>indieweb</span></a> <a href=\"https://mastodon.social/tags/posse\" class=\"mention hashtag\" rel=\"tag\">#<span>posse</span></a> <a href=\"https://mastodon.social/tags/webmention\" class=\"mention hashtag\" rel=\"tag\">#<span>webmention</span></a> <a href=\"https://mastodon.social/tags/webdev\" class=\"mention hashtag\" rel=\"tag\">#<span>webdev</span></a> </p><p><a href=\"https://squeaki.sh/p/being-part-of-the-indieweb\" target=\"_blank\" rel=\"nofollow noopener noreferrer\" translate=\"no\"><span class=\"invisible\">https://</span><span class=\"ellipsis\">squeaki.sh/p/being-part-of-the</span><span class=\"invisible\">-indieweb</span></a></p>",
      "reblog": null,
      "application": {
        "name": "Web",
        "website": null
      },
      "account": {
        "id": "111986074552457300",
        "username": "steffoz",
        "acct": "steffoz",
        "display_name": "Stefano Verna",
        "locked": false,
        "bot": false,
        "discoverable": true,
        "indexable": true,
        "group": false,
        "created_at": "2024-02-24T00:00:00.000Z",
        "note": "<p>Founder of DatoCMS</p>",
        "url": "https://mastodon.social/@steffoz",
        "uri": "https://mastodon.social/users/steffoz",
        "avatar": "https://files.mastodon.social/accounts/avatars/111/986/074/552/457/300/original/0f3dbfe9f32eebf6.png",
        "avatar_static": "https://files.mastodon.social/accounts/avatars/111/986/074/552/457/300/original/0f3dbfe9f32eebf6.png",
        "header": "https://files.mastodon.social/accounts/headers/111/986/074/552/457/300/original/0c3ca789b56783e4.png",
        "header_static": "https://files.mastodon.social/accounts/headers/111/986/074/552/457/300/original/0c3ca789b56783e4.png",
        "followers_count": 9,
        "following_count": 36,
        "statuses_count": 18,
        "last_status_at": "2024-03-28",
        "hide_collections": null,
        "noindex": false,
        "emojis": [],
        "roles": [],
        "fields": [
          {
            "name": "Personal blog",
            "value": "<a href=\"https://squeaki.sh\" target=\"_blank\" rel=\"nofollow noopener noreferrer me\" translate=\"no\"><span class=\"invisible\">https://</span><span class=\"\">squeaki.sh</span><span class=\"invisible\"></span></a>",
            "verified_at": "2024-03-10T15:17:42.108+00:00"
          },
          {
            "name": "Work",
            "value": "<a href=\"https://www.datocms.com\" target=\"_blank\" rel=\"nofollow noopener noreferrer me\" translate=\"no\"><span class=\"invisible\">https://www.</span><span class=\"\">datocms.com</span><span class=\"invisible\"></span></a>",
            "verified_at": null
          }
        ]
      },
      "media_attachments": [],
      "mentions": [],
      "tags": [
        {
          "name": "indieweb",
          "url": "https://mastodon.social/tags/indieweb"
        },
        {
          "name": "posse",
          "url": "https://mastodon.social/tags/posse"
        },
        {
          "name": "webmention",
          "url": "https://mastodon.social/tags/webmention"
        },
        {
          "name": "webdev",
          "url": "https://mastodon.social/tags/webdev"
        }
      ],
      "emojis": [
        {
          "shortcode": "pensive_party_blob",
          "url": "https://files.mastodon.social/custom_emojis/images/000/086/980/original/31fd04ff8be27277.png",
          "static_url": "https://files.mastodon.social/custom_emojis/images/000/086/980/static/31fd04ff8be27277.png",
          "visible_in_picker": true
        }
      ],
      "card": {
        "url": "https://squeaki.sh/p/being-part-of-the-indieweb",
        "title": "Being part of the IndieWeb",
        "description": "It’s hard to believe, but there was a time in the web era when social media didn’t exist. Imagine this: the year is 2000. Apple is absolutely crushing it with their latest iPods...Microsoft was......",
        "language": "en",
        "type": "link",
        "author_name": "",
        "author_url": "",
        "provider_name": "squeaki.sh",
        "provider_url": "",
        "html": "",
        "width": 1200,
        "height": 600,
        "image": "https://files.mastodon.social/cache/preview_cards/images/092/322/880/original/f74ec09b5a4b9c03.png",
        "image_description": "",
        "embed_url": "",
        "blurhash": "UFR:KR4.tRxu?DbHofRj.A-;IoM{~URkM{xu",
        "published_at": null
      },
      "poll": null
    }
  ],
  "descendants": [
    {
      "id": "112167993661516620",
      "created_at": "2024-03-27T13:55:54.849Z",
      "in_reply_to_id": "112167934611297209",
      "in_reply_to_account_id": "111706433827905439",
      "sensitive": false,
      "spoiler_text": "",
      "visibility": "public",
      "language": "en",
      "uri": "https://mastodon.social/users/steffoz/statuses/112167993661516620",
      "url": "https://mastodon.social/@steffoz/112167993661516620",
      "replies_count": 0,
      "reblogs_count": 0,
      "favourites_count": 1,
      "edited_at": null,
      "content": "<p><span class=\"h-card\" translate=\"no\"><a href=\"https://social.lol/@amerpie\" class=\"u-url mention\">@<span>amerpie</span></a></span> cool! One more reason to write a more technical post about it :)</p>",
      "reblog": null,
      "application": {
        "name": "Mastodon for Android",
        "website": "https://app.joinmastodon.org/android"
      },
      "account": {
        "id": "111986074552457300",
        "username": "steffoz",
        "acct": "steffoz",
        "display_name": "Stefano Verna",
        "locked": false,
        "bot": false,
        "discoverable": true,
        "indexable": true,
        "group": false,
        "created_at": "2024-02-24T00:00:00.000Z",
        "note": "<p>Founder of DatoCMS</p>",
        "url": "https://mastodon.social/@steffoz",
        "uri": "https://mastodon.social/users/steffoz",
        "avatar": "https://files.mastodon.social/accounts/avatars/111/986/074/552/457/300/original/0f3dbfe9f32eebf6.png",
        "avatar_static": "https://files.mastodon.social/accounts/avatars/111/986/074/552/457/300/original/0f3dbfe9f32eebf6.png",
        "header": "https://files.mastodon.social/accounts/headers/111/986/074/552/457/300/original/0c3ca789b56783e4.png",
        "header_static": "https://files.mastodon.social/accounts/headers/111/986/074/552/457/300/original/0c3ca789b56783e4.png",
        "followers_count": 9,
        "following_count": 36,
        "statuses_count": 18,
        "last_status_at": "2024-03-28",
        "hide_collections": null,
        "noindex": false,
        "emojis": [],
        "roles": [],
        "fields": [
          {
            "name": "Personal blog",
            "value": "<a href=\"https://squeaki.sh\" target=\"_blank\" rel=\"nofollow noopener noreferrer me\" translate=\"no\"><span class=\"invisible\">https://</span><span class=\"\">squeaki.sh</span><span class=\"invisible\"></span></a>",
            "verified_at": "2024-03-10T15:17:42.108+00:00"
          },
          {
            "name": "Work",
            "value": "<a href=\"https://www.datocms.com\" target=\"_blank\" rel=\"nofollow noopener noreferrer me\" translate=\"no\"><span class=\"invisible\">https://www.</span><span class=\"\">datocms.com</span><span class=\"invisible\"></span></a>",
            "verified_at": null
          }
        ]
      },
      "media_attachments": [],
      "mentions": [
        {
          "id": "111706433827905439",
          "username": "amerpie",
          "url": "https://social.lol/@amerpie",
          "acct": "amerpie@social.lol"
        }
      ],
      "tags": [],
      "emojis": [],
      "card": null,
      "poll": null
    }
  ]
}