Send webmentions

[chrysn]

Is your proposal related to a problem?

Without going full ActivityPub (not sure if it'd work even then), I can't tell from my website whether it has been talked about on lemmy.

Getting this information would be useful, as then the (mostly static) website could, rather than employing any local or centralized "comments" tool, say something like "Post about this on the fediverse, eg. by posting it on Lemmy", and then have below a list of all the threads talking about this particular page.

Describe the solution you'd like

Implementing the Webmention specification would just do this:

Whenever a post is created about a page (possibly also when it's just linked to in formatted text, I'm not sure whether that makes sense), the Lemmy server would load the page (it probably does so anyway already to get title and stuff), look at whether it has a rel="webmention" link outgoing (in Link headers, or a <{link,a} href="..."> link), and if so POSTs a short x-www-form-urlencoded thingy there.

There's probably a few details I'm missing (being new to the tool myself), but in the end it should rather be straightforward.

Describe alternatives you've considered

There's a lot of technologies that led up to webmentions -- pingback, refback, talkback, trackback -- but webmentions is what made it to W3C Recommendation and generally looks sound.

The static page could, instead, go full ActivityPub, participate in the fediverse and look for mentions -- but a) I don't even know if that works without the page setting itself up, and b) is a lot more complex than webmentions, which a page can even do just in JavaScript by referring to https://webmention.io/ and evaluating it at runtime (although there may be reasons not to do that, cf. https://martymcgui.re/2020/07/15/what-we-talk-about-when-were-talking-about-webmentions/ on "Just let JavaScript do it!").

I could of course link to something like https://commentpara.de/ -- but Lemmy lends itself to having an actual discussion,

Additional context

This has been requested for Reddit, but rather ignored there.

How I tested it doesn't work already

(for I don't have a website running that'd actually make use of it yet)

I checked the source code for any mention of "webmentions", "pingback", "linkback", "refback", "talkback" and "trackback" (the technologies available for this mentioned on https://github.com/pelican-plugins/linkbacks/), and none of that showed up.

Also, the post at https://lemmy.ml/post/41713 mentions https://martymcgui.re/2020/07/15/what-we-talk-about-when-were-talking-about-webmentions/ which uses the webmention.io backend and thus would be visible at https://webmention.io/api/mentions.jf2?target=https://martymcgui.re/2020/07/15/what-we-talk-about-when-were-talking-about-webmentions/ -- but nothing there indicates that lemmy has ever pinged it.

--- Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/96588648-send-webmentions?utm_campaign=plugin&utm_content=tracker%2F126011972&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F126011972&utm_medium=issues&utm_source=github).

[Nutomic]

We use iframely to get info about linked sites, and it has an endpoint to get the rel field among other things: https://iframely.com/docs/iframely-api

So this is gonna need a new method in crates/utils/src/request.rs which calls that endpoint and extracts the value (because right now we only use the oembed endpoint). And then a change in crates/api/post.rs::CreatePost to make the webmention request itself.

Contributions welcome, we probably wont be able to work on this anytime soon.

[chrysn]

I'm open to giving it a try, but how would I test this locally? The publicly available test API does seem to support arbitrary links including webmention (checked on http://debug.iframely.com/?uri=https%3A%2F%2Fwebmention.io on origin data), but how can I (now or even when playing around with the code) verify it works with the actually used API endpoint? Looks like I'd need to populate the iframely_url config value – do I need to get a subscription with Iframely, and even then where does the token go in? (format!("{}/oembed?url={}", Settings::get().iframely_url, url); seems to indicate it's not in the iframely_url)

And implementation details apart, would this be active by default on lemmy instances? (It really only makes sense if it is, so we may want to have that discussion beforehand. IMO it's OK because it's not like the information can't be spidered off just as well, so it could be on by default unless the instance kind of completely locks itself down).

[Nutomic]

You should be able to test it with the setup in docker/dev/docker_update.sh. I'm also not sure if this should be a default, in fact thats what I was gonna ask. Does it make sense to send webmentions always, or would it make sense to add a config option or something?

[chrysn]

You should be able to test it with the setup in docker/dev/docker_update.sh.

That looks scary -- but I'd postpone actual work on an implementation until we know whether this is really desirable for Lemmy first anyway.

Does it make sense to send webmentions always, or would it make sense to add a config option or something?

It only makes sense as a default. If website creators can't rely on most Lemmy instances to have this on, they won't say "Want to add to this? Just post about it on Lemmy" (because it'd deteriorate into a "provided your instance admin has webmentions enabled, and if not tell them they can enable it like this") -- they'd just point somewhere else (say, lobste.rs, which implemeted this a while ago https://github.com/lobsters/lobsters/issues/383 -- I don't want to play services against each other here, I was just looking at where I'd go if Lemmy wouldn't support it, and that's what looks usable after going through links on https://indieweb.org/Webmention).

IDK if there's ways to run Lemmy instances in some private mode -- where posts are only accessible to authenticated users; for such cases, it probably makes sense to add an option to disable it, but if it's off by default, barely a site admin will turn it on (just laziness), and then if a webmention happens it's maybe a pleasant surprise but nothing that'd be relied on.

[Nutomic]

I see, as long as there are no negative effects from webmentions then I'm fine with having it as default. So far all posts and communities are public, so you dont have to worry about that. We are gonna add private communities later, and will have to keep in mind to disable webmentions then (so maybe just mention it in a code comment).

[Nutomic]

I checked and there is actually a webmentions crate. So fixing this issue should be as simple as calling that crate from CreatePost, then running Lemmy locally to test if it sends the webmention as expected. Would you like to give that a try?

[chrysn]

Thanks a lot for implementing this!