Instance agnostic links

ivanjermakov commented 1 year ago

Problem

As of now, only direct HTTP links are supported, e.g. https://lemmy.world/post/1 for posts and https://lemmy.world/u/ruud for users.

This is inconvenient when one would like to stay on a single instance following links in commens/posts.

Solution

Introduce a special format for instance agnostic links (called Lemmy links) that will be opened in the current instance:

for posts: #1@lemmy.world
for users: @ruud@lemmy.world
for communities: !asklemmy@lemmy.ml
for comments: ~1@lemmy.world

This will open page at https://<instance>/post/1, https://lemmy.world/u/ruud, https://lemmy.world/c/asklemmy@lemmy.ml, https://lemmy.world/comment/1 respectively. It will not break existing links functionality as direct HTTP links are still supported.

In order to easily create Lemmy links, new link button can be introduced (next to the fedilink). It will copy Lemmy link into the clipboard.

Discuss

Actual link format might be something else, but the general idea is clear. It must include:

Instance (domain)
Object (user, comment, post)
Object id

I like the idea of having a symbol prefix for each object, (# for posts, @ for users, ! for communities and ~ for comments). This way, format is (#@!~)<id>@<instance>.

Additional context

If this request gains traction, I might be able to implement the feature myself.

Klobenda commented 1 year ago

:+1: Server-side detection-and-rewrition of community/user mentions (!community@host and @user@host) to turn them into local links would be awesome - though today if a community or user hasn't been federated to your instance it won't be accessible at c/{community} etc, going to those pages should probably kick off a federation attempt.

ivanjermakov commented 1 year ago

I'm new and haven't seen the code yet, keep this in mind. I'm not clear how ids for subjects from another instances are assigned. If you open a post from another instance it has two ids: id on the instance it was posted on and id on the current instance. Can someone explain how ids are assigned and is there a way to tell (client-side or redirect) what id shold be used?

Suppose example:

Being on lemmy.world, follow link @lemmy.ml/post/1165318
At this point in time, client has no information where to find this post in local instance (and it also might be accessible, as @Klobenda mentioned)
Clicking link should redirect to https://lemmy.world/post/12774

ivanjermakov commented 1 year ago

@Klobenda it should be possible to check if community is accessible and the link is valid during page construction. If it is not it can be highlighted somehow on the UI. But this might require additional DB check which is not ideal.

chaorace commented 1 year ago

What's the point in overcomplicating it?

We already have a pattern for extending local syntax into agnostic syntax established for community/user URLs: /c/meta => /c/meta@lemmy.ml & /u/nutomic => /u/nutomic@lemmy.ml. Why don't we just strictly follow the same convention for the other URL types?

/post/12345 => /post/12345@lemmy.ml
/comment/123456 => /comment/123456@lemmy.ml

Yeah, sometimes federation gets in the way and you can't resolve a link locally, but that already happens for external community links and that was deemed an acceptable outcome. Keeping the syntax unified means that we can solve the fetching/resolution problem once instead of thrice. Even if the problems can't be fixed right away, it's still useful to at least implement the syntax so that mobile clients can get the memo and have support ready.

Nutomic commented 1 year ago

Each Activitypub object (post, comment, user, community) has an ID. In case of posts this looks like https://lemmy.world/post/1 with the domain where the creator is registered. When another instances fetches the post, it inserts it in the db and renders it as a link with the db post.id column value, eg https://lemmy.ml/post/3. So the problem is that the url id is exactly the same as the db id, which naturally differs between instances. A cleaner solution for this would be random IDs as described in #1101. This is what Peertube does for example.

chaorace commented 1 year ago

That makes sense, I'll comment my thoughts about that under the linked issue.

What you say does raise a new question relevant to this dicussion, however: why the schism between community/user URLs & post/comment URLs? What's the underlying motivation behind strictly marrying the backend ID & Fediverse ID in the latter case?

ivanjermakov commented 1 year ago

necropola commented 1 year ago

lemmy-ui-0.18-0 kinda solved this for users and communities through LemmyNet/lemmy-ui#1462, but doing the same for posts and comments requires global UUIDs for those objects.

Dakkaron commented 1 year ago

Tbh, I wouldn't go for UUID, since that makes for super long urls.

Each post already has the info of where it's true copy lies.

So e.g. for a post that originated at lemmy.ml/post/123 and is replicated on feddit.de/post/321 this could be done:

The post could be accessed on feddit.de as:

feddit.de/post/321
feddit.de/post/123@lemmy.ml

This should be doable without much db changes.

And if you get a 404 on 123@lemmy.ml, because it is not locally replicated, it would be easy to show a link to the original post at lemmy.ml/post/123.

It would even be quite easy to add a button that searches for the community and replicates it to the local instance.

KaKi87 commented 1 year ago

Hello,

I've just released Fediverse redirector, a userscript that, among other things, automatically redirects all Lemmy community/post links to your favourite instance.

May it make your interactions easier. :)

marsara9 commented 1 year ago

With https://github.com/LemmyNet/lemmy/pull/3716 I have an idea on something that can be done here. (Thinking out loud but please provide feedback)

Whenever a new post or comment gets added to the DB, either from someone submitting locally or via federation, prior to adding said comment/post to the DB, look in the body for any URL containing https://.../post/### (will use a proper Regex but I don't remember the rules for domain names off the top of my head). But any matches found will use search_query_to_object_id_local(...) to attempt to convert the URL to a local object ID. If found the URL would be replaced in the content before inserting into the DB.

Now this relies on the URL being the ap_id of the post or comment in order to successfully resolve. If possible an additional step prior to calling search_query_to_object_id_local(...) would be to call api/v3/post/ on the same domain as the URL and fetch the ap_id that way. Then feed that back into the new search method. This way ANY post that is federated with the local instance should resolve.

Finally if we want to go even further, we could use the non-local version of that search method to kickoff a remote fetch? But I fear even the API call could severely hinder performance, let alone a call to search_query_to_object_id(...).

Another thought I had was to do this in lemmy-ui but I'd need to find a way to cache the API call and resolve_object calls somehow. And even then, how long do you cache them for? Could this be done at the network layer itself?

Nutomic commented 1 year ago

@marsara9 You suggestion makes sense. It should definitely be implemented in the Lemmy backend for performance reasons, and so that all apps can benefit from it.

marsara9 commented 1 year ago

Just to be clear I've got essentially 3 proposals:

1) only do a local search:

A new post or comment is about to be added to the DB
Do a local lookup as if any URLs that match the Regex are in the post or comment body.
Replace said link with the new URL, if found.
If nothing is found either because that link wasn't the ap_id or because that linked post or comment hasn't federated then the original link is left intact.

2) do a remote lookup to get the ap_id first:

A new post or comment is about to be added to the DB
Using the domain from the link, make a call to api/v3/post or api/v3/comment (depending on which Regex matched) to determine the ap_id.
Do a local lookup on those ap_ids to find their internal id.
Replace said link with the new URL, if found.
If nothing is found either because that linked post or comment hasn't federated then the original link is left intact.

3) kickoff a remote fetch so that all URLs will convert.

A new post or comment is about to be added to the DB
Using the domain from the link, make a call to api/v3/post or api/v3/comment (depending on which Regex matched) to determine the ap_id.
Do a remote lookup on those ap_ids to find their internal id, kicking off a federation request if the ap_id doesn't exist in the local DB.
Replace said link with the new URL.
If nothing is found either because that link wasn't valid then the original link is left intact.

(For any of the above if the domain in the link already points to the current instance this whole process is skipped)

The main downside to doing this in the backend for options 1/2 is that if the linked post or comment hasn't been federated yet, then the link will forever take the user to the remote instance. However, in theory any referenced link should be far enough in the past that it should be federated already.

Die4Ever commented 1 year ago

I also think instance agnostic links should work in the URL.

Like if a user wants to share a link with someone, they're likely going to just copy-paste from their address bar.

If I share this link https://programming.dev/post/2059750

Currently you would have to open that link, then get the permalink, then open your home instance and search for the permalink...

If instead my address bar said https://programming.dev/post/2174573@lemmy.blahaj.zone

Then an inexperienced Lemmy user could just copy-paste out of their address bar and share the link on Reddit or whatever. Another Lemmy user should be able to paste that link into their instance's search directly without modifications. And because the real instance name of the post is in the URL, your home instance won't actually need to contact programming.dev to do the search.

For a smoother transition and to retain the local post ID, you could use a URL like

https://programming.dev/post/2059750/2174573@lemmy.blahaj.zone

So loading the page will use the 2059750 for the local post ID and ignore the rest, but searches would use the 2174573@lemmy.blahaj.zone at the end and ignore the local ID

phiresky commented 1 year ago

There's also protocol handlers.

On your home instance, lemmy-ui (or your favourite app would register itself as a web+lemmy: protocol handler. In JS this looks like navigator.registerProtocolHandler("web+lemmy", "https://lemmy.world/resolve?url=%s", "Lemmy cross-instance link handler")

On firefox, this results in a prompt like this:

On chrome like this:

Now, all links to web+lemmy:... will go to your home instance (or app on android).

Every lemmy-ui would have a button "Open on my local instance". The button is a href to web+lemmy:https://<post/comment/community/user url>.

Edit: I think there's multiple sparate issues being discussed in this issue. I'm making it worse. Probably would make sense to split it out more or make it more generic

Die4Ever commented 1 year ago

There's also protocol handlers.

I think your idea is really good, but yeah probably a separate issue. Also I'm guessing that doesn't carry over through copy pasting links

Die4Ever commented 6 months ago

for posts: #1@lemmy.world
for users: @ruud@lemmy.world
for communities: !asklemmy@lemmy.ml
for comments: ~1@lemmy.world

Another one that would be good is the ability to mention an instance, and the link would open their list of communities? Maybe something like &retrolemmy.com, but now that we have Remote Follow maybe this isn't so necessary as you could just link to their communities page remotely

LemmyNet / lemmy