Server-side plugins / extensions

[dcx]

Requirements

• [X] Is this a feature request? For questions or discussions use https://lemmy.ml/c/lemmy_support
• [X] Did you check to see if this issue already exists?
• [X] Is this only a feature request? Do not put multiple feature requests in one issue.
• [X] Is this a backend issue? Use the lemmy-ui repo for UI / frontend issues.

Context: I'm not an active contributor, but I run an instance for a country community, and had an idea which I felt might be worth bouncing around. I hope this is okay to do here.

Is your proposal related to a problem?

So I've been running into lots of random little frictions that are very specific to my use case (list below). I feel I could solve these by maintaining a private fork. But I've been thinking about whether this is the best way to handle this situation.

Could it make sense to provide support for modular server-side plugins or extensions, so that Lemmy instance admins can easily tailor Lemmy to their specific use cases, and share their customisations with each other? Rationale:

• I've noticed that instances cluster into several broad use cases, e.g.: for specific geographies, languages, hobbies, industries, media (single TV shows or games), content types (NSFW, podcasts).
• This means there are features which might drastically improve the experience for a couple of use cases, but none of the others. So they probably won't exist in vanilla Lemmy for a long time, if ever, due to poor cost-benefit and bloat.
• This might also allow way more developers to work on Lemmy, with minimal extra load to the core team. There's lots of non-professional developers out there who are capable of writing a small plugin given some safety rails (see: automatic1111, WordPress). And over time this might become a feed of good ideas/code back into the main codebase.

Describe the solution you'd like.

• Anything quick and easy: This might not need to be invasive or hard to support. Adding a few text rendering callbacks and customization points (e.g. a second user flair spot?) might create a lot of value, perhaps with zero database impact (we could require extensions to store their data separately)

Describe alternatives you've considered.

• Maintaining a private fork: This doesn't seem like good ROI for the sake of small quality-of-life features. But if I could write something which benefits lots of other instance admins, and has interfaces to Lemmy which I know won't change drastically from version to version, this changes the calculus quite a bit!
• UI-only or browser extensions: Since users use our instance via multiple frontends, I think extensions primarily need to happen on the backend, and operate on the content feed itself. (But many might need to target lemmy-ui as well)

Additional context

Here are a bunch of random examples OTOH for my instance, running a country instance for Malaysia. I haven't yet thought deeply about implementation via plugins, so please excuse me if some of these are demanding or impossible. I'm thinking of this more as an intuition pump.

• I'd like to be able to flag posts as Politics, and completely hide them from my feed. Community-level segregation for politics doesn't work, as country instances are evolving towards exposing a main !Country for broader Lemmyverse consumption (lemmy.ca, aussie.zone, feddit.uk all do so). Half our users want politics on their feeds, and half hate it.
• Alternatively, I'd like to be able to designate a "virtual community" for !Country which compiles posts from several real communities into a single RSS feed.
• Malaysians cluster into distinct blocs with different first languages (English, Chinese, Malay, Tamil). Seeing lots of unreadable content is very alienating, so I'd like to let users select languages at signup, with our specific national languages sorted to the top of a list.
• Malaysians are often multilingual; it could be nice to have a Google Translate clicky for post text which is in non-primary languages.
• I'd like some specific UI / handling around hashtags for a very niche use case: The Southeast Asian Fediverse has apparently standardised on hashtags for content discovery.
• I'd like to display some fairly specific community growth metrics in the sidebar for moderators, which reflect how we're driving traffic from our subreddit. We can generate charts or text, but could use an injection point.
• I'd like to designate a list of subs as not appearing on the Local feed by default.

I hope this idea isn't terrible, and I haven't missed anything obvious! (I've tried searching lemmy and lemmy-ui, and !lemmy_support, and didn't find anything)

[gennitdev]

I'm interested in the idea of plugins as well. Here are some other plugin ideas that I have been thinking about:

• I have a project that lets people search and filter in-person events by distance, tags and other parameters. I intend to eventually integrate it with the Fediverse. If I could make an in-person events plugin for Lemmy, it might be cool if a Lemmy server could see events on my project's server.
• It might be cute and fun if some Lemmy servers had a plugin for avatar customization that would let you change the hair and clothing, similar to reddit. If some of the avatar's clothing items cost money in the real world, it would be a way to pay the web hosting bills of the server and extend its life.
• A plugin might help with grassroots fundraising. Picture a progress bar on the sidebar that depicts a fundraising goal for the server or a community within it. A plugin would help integrate the fundraising badge or progress bar with other sites like Patreon and GoFundMe, with the ultimate goal of supporting the longevity of the community.

[gwbischof]

I am interested in this also, and I can help contribute code

[Nutomic]

A plugin system would definitely be good to have. I think wasm is the preferred approach for plugins in Rust, because it allows you to write plugins in many different languages. In any case it seems quite complicated to implement, you can try to get a proof of concept working.

I'd like to be able to flag posts as Politics, and completely hide them from my feed. Community-level segregation for politics doesn't work, as country instances are evolving towards exposing a main !Country for broader Lemmyverse consumption (lemmy.ca, aussie.zone, feddit.uk all do so). Half our users want politics on their feeds, and half hate it.

The backend has an endpoint /api/v3/community/hide for this purpose. Note that its not implemented in lemmy-ui so you have to call it manually.

Malaysians cluster into distinct blocs with different first languages (English, Chinese, Malay, Tamil). Seeing lots of unreadable content is very alienating, so I'd like to let users select languages at signup, with our specific national languages sorted to the top of a list.

Would be reasonable to implement this directly in lemmy-ui. Default languages could be taken from site languages.

Malaysians are often multilingual; it could be nice to have a Google Translate clicky for post text which is in non-primary languages.

Also reasonable as a builtin feature.

I'd like some specific UI / handling around hashtags for a very niche use case: The Southeast Asian Fediverse has apparently standardised on hashtags for content discovery.

This would be very complicated, I dont think hashtags can federate well with the way Lemmy works. And changing federation through plugins sounds like a bad idea because that can easily break things.

I'd like to display some fairly specific community growth metrics in the sidebar for moderators, which reflect how we're driving traffic from our subreddit. We can generate charts or text, but could use an injection point.

lemmy-ui has an env var to include custom html in the page header for tracking scripts.

I'd like to designate a list of subs as not appearing on the Local feed by default.

Same as the first point, no?

[erlend-sh]

flair by @tristanisham and @ornato-t seems like a useful reference for a would-be plugin.

I wonder if what they’ve made as a microservice could instead be done as an Extism plugin.

https://dylibso.com/blog/why-extism/

[ornato-t]

Hey there. So, we built flair as a Rust crate, with an API wrapper to run it as a microservice.

While neither I nor other people on the team (to my knowledge) have any experience with WASM and Extism, I don't think it would be hard to convert our existing service to that. If the Lemmy backend ended up supporting WASM plugins, I'd be happy to port flair there; it would easily jump at the top of our priority list. Looking forward to this!

[Nutomic]

@ornato-t We dont have any plans to work on this. That would be very difficult anyway because we dont know which hooks might be useful for plugins, or how things should be structured. I think it makes more sense if you give it a try yourself to hack something together. Doesnt have to be pretty, its just to get an idea how it can work.

[tristanisham]

I've got some experience compiling Rust to WASM, but I'm concerned with WASMs limitations? Flair relies on accessing a database (currently on disk, but in the future, it can access the same database as Lemmy). As far as I'm aware WASM is really good for secluded data manipulation. Not for manipulating data on the disk?

[ornato-t]

I understand @Nutomic. A plugin system is something I'd love to see as a consumer, but that would probably require tons of work on your end.

Anyway, right now we are about to finish building flairs to run as a REST API, separated from the Lemmy backend and called from a modified version of the Lemmy UI. Later on I would also like to fork Lemmy and include our user flair service within it. Possibly also make a PR or at least propose to include it within Lemmy proper, but this is probably not the right place to discuss this.

[erlend-sh]

As far as I'm aware WASM is really good for secluded data manipulation. Not for manipulating data on the disk?

Not sure if it applies to your use case, but there’s this: https://dylibso.com/blog/pg-extism/

[tristanisham]

Oh lord, that looks like a lot of work. Could you also, if it's not too much trouble, send a repo that has implemented something from rust into this kind of extension?

Would this be easier if there were some extension trait built into Lemmy, events extensions could register, and the ability to link in a .so?

Worst case they could all be bundled as separate executables Lemmy itself manages,though that'd be a big DX hit.

[erlend-sh]

I’m not familiar with it beyond the high level details, so you’d have to ask on the Extism discord for more information.

[nilslice]

Hi, I'm from the Extism team and would be happy to help if folks are interested in using Extism.

Integrating Extism into Rust is a really straightforward option, here's a two-line example that loads some wasm from disk and executes a function from within it: https://github.com/dylibso/modsurfer/blob/e54efeff8e9d96f6e2ca9b671eeaac4d522366c8/validation/src/lib.rs#L482-L483

I'd have to take a look at the Lemmy source more closely -- would anyone be able to point me to some location in source where you'd consider adding an extension point to call wasm code?

[Nutomic]

Regarding user flairs I think it makes more sense to implement it directly in Lemmy rather than an extension. However its a complex feature so I suggest you write an rfc to figure out how federation, moderation and other aspects can work. Otherwise you can develop it in a fork/pull request and we figure out along the way how to get it into a mergable state.

[tristanisham]

Okay, let Tommaso and I read the RFC documentation and get back to you. We are definitely interested in directly building flairs into Lemmy. 

[Nutomic]

Let me know if anything is unclear about the RFC process. Its brand new and copied from the Rust project, so it might need some adjustments.

[AgentScrubbles]

@Nutomic I was just thinking about extensions and thought of this possible solution, I was reading a thread somewhere on Lemmy and came thought about the already increasing problem of spam-my bots in comments. Things like Piped and tldr are great, but everyone kinda agrees that they are just kinda comment fillers. If a plugin system was built, I was thinking it could be interesting to have a separate table in the DB, something like post_plugin, that would be laid out somewhat like:

| id | post_id | plugin_id | markdown                                                                                   | 
| 1 | 123       | 42          | Here's your Piped link [Piped Video Link](https://foo.bar/video/1) |

Then, either in the Post GET or as a separate API, you could grab any extensions for a specific post, and underneath each post we could have something (hopefully standardized), with common extensions. So posts could be something clean like

[ Title ]
[ Post Body ]

[ Piped Link ]
[ Expandable TL;DR ]
[ Any other extensions, Spam detector, Whatever developers choose. ]

For each one I thought about maybe a standardized badge, something like > Piped Link that's expandable to show the Markdown, where any icon/plugin name could be stored in the plugins table and retrieved at the same time.

How plugins are implemented and called I'll stay out of for now, but I thought this might be a clean way to organize them in the DB, and then a really nice clean way to display them..

I was going to make this a separate feature request, but finding plugins here I think I'll just pitch my idea here. I know this would be a ton of work, but if something like server side plugins are already being thought of, I thought I'd throw in this architecture/design idea as a way to standardize them. If you would prefer this to be it's own request, I'd be happy to make one, or if you don't like it hey that's okay too, I just had this idea about the UI of it and thought about sharing.

[Nutomic]

This post on /r/rust has some interesting information about Rust plugins. The gist of it seems to be that wasm plugins are more performant and allow writing plugins in different languages, but are also much more complicated. Extism is an example. On the other hand there are scripting languages such as mlua which are easier to use, but less performant.

[db0]

My preference would be for wasm because this will allow any developer with programming language knowledge to "scratch their own itch", (so to speak.) and then help others do so likewise.

Looking at the example posted earlier, integrating wasm plugins doesn't look like a particularly complex procedure. All it would need is to decide where to add hooks for them, and then the deployment mechanism which would easily be handled via ansible initially. Of course, not being a rust dev, I don't quite know if the difficulty is with compiling support for extism into the source or something else.

Can you elaborate on which aspect of adding something like extism would be too complicated currently to help me understand?

[erlend-sh]

Kitsune has made some significant headway here:

• https://github.com/kitsune-soc/kitsune/issues/160
• https://github.com/kitsune-soc/kitsune/pull/490
• https://hackmd.io/@aumetra/rkXOLyasp

The MRF system uses the WASM Component Model to provide a consistent, simple, and performant ABI between the host and guest.

[db0]

You think that progress could be used as a template to port this functionality to lemmy?

[erlend-sh]

I’m not sure about the extent to which this work can be ported to Lemmy, but I think it clearly demonstrates the applicability of WASM for server-side plugins. cc @aumetra

[aumetra]

You think that progress could be used as a template to port this functionality to lemmy?

Definitely can. The current infra Kitsune implemented is mostly geared towards rewriting activities, so it's not too useful for things like aggreating custom feeds, etc. But it can be used to mark things as sensitive or even drop activities entirely (i.e. for politics posts, as mentioned in the original issue text).
But it can definitely be used as a point for inspiration for Lemmy to develop its own plugin API with more extensive functionality, usable outside of the realm of rewriting activities.

(Of course Lemmy can also implement the MRF API, just to have some interop with WASM MRFs)

I need to formulate the actual API I implemented into the MRF FEP, then there might be a bigger potential for adopting this into more software projects.

---

Edit: Also, about potential configuration UI for these plugins, the MRF manifest includes something rather nice.
Configuration is passed to the module, and the manifest can optionally include a JSON Schema for the configuration. Developers can use this to automatically generate nice looking forms with radio buttons, fields, validations, etc. to configure the modules.

[Nutomic]

Can you elaborate on which aspect of adding something like extism would be too complicated currently to help me understand?

I dont have any experience with this, its just what I read in the linked Reddit thread. Anyway contributions for this are welcome.

[nilslice]

Is there anything we (the @extism team) can help with?

for anyone interested in working on a plug-in system for this, please feel free to reach out on discord & we can find ways to contribute!

https://extism.org/discord

[Nutomic]

I have created a proof of concept for Lemmy plugins. If you are interested in this feature, please have a look and comment about the specific plugin hooks that you need.

[bkil]

For some inspiration, see also:

• https://github.com/friendica/friendica-addons
• https://github.com/friendica/deprecated-addons
• https://framagit.org/hubzilla/addons

[Nutomic]

This PR adds documentation on how to write Lemmy plugins