Switching backends

[ClearlyClaire]

For multiple reasons (poor performances, high resources usage, weird handling of ActivityPub), I'm growing increasingly uncomfortable running Mastodon's backend.

This issue is here to track some of the stuff we could do to make moving away from the current backend easier.

• [ ] Making the front-end easier to use with different backends
  • [ ] Simpler and/or better documented build process (if possible, without depending on Ruby stuff) It is possible to build the front-ends without installing Ruby stuff. yarn run build:production should do the trick. If applicable, environment variables such as CDN_HOST have to be set first, though. It may be valuable to enable selecting the front-end to build. TODO: check if passing --config config/webpack/production.js changes anything?
  • [ ] Better documented API
  • [ ] Better API versioning, dynamically checking API capabilities, failing gracefully on API failures, etc.
• [ ] Providing export scripts so that data can be migrated to other backends This involves collaborating with developers of other ActivityPub servers. The migration itself is also probably going to be tricky because we at the very least need to keep ActivityPub object identifiers valid. Related pleroma issue: https://git.pleroma.social/pleroma/pleroma/issues/162
• [ ] Improving account migration mechanisms, which is useful in all cases, but especially so if exporting to another backend turns out to not be possible

[jalcine]

This is a huge undertaking. What are some slower rewrite attempts that can be taken? Something that comes to mind is turning this into a gateway of sorts that has bits and parts exported to Rails engines that one could replace with other technologies (or more performant code).

[rhaamo]

Mastodon isn't very modular so I don't think it's possible at all to just replace /some parts/. I'm pretty sure making some sort of "gateway" for various micro-services would be way more painful that just making the frontend dissociated from the backend.

Splitting front/back isn't that hard, it has been done for tootsuite one. The migration here is the hardest in the sense it requires good knownledge of ActivityPub and lot of time to match features from both sides.

[ClearlyClaire]

@jalcine I don't think the resource usage can be improved much without ditching RoR entirely. As for performance, I'm thinking about switching from the worker threads-based server puma to the asynchronous coroutines-based falcon but there are a lot of reasons why that may not be the best solution. I'm not too sure what other improvements we could make within the existing framework, but I think they would also each involve a lot of work.

[jalcine]

Gotcha; definitely just spitting out thought into the abyss.

The thoughts around splitting the front-end and back-end sounds like it'd be more so to make the server focus more on processing business logic vs coupling view, presentation and business all in one spot.

That said, if modularization and/or decoupling is hard, what other choices outside of switching server implementations can help?

[jalcine]

Ah haha, posted too soon. Is there a set of benchmarks that could test the switching of Web servers for GlitchSoc/Mastodon?

[rhaamo]

The thoughts about splitting front and back is more to be able to use whatever backend we want as long as it implements mastodon APIs.

Currently it's possible to use tootsuite mastodon frontend with pleroma as a backend with some patchs.

[rhaamo]

@jalcine there is actually no benchmarks about using anything other than Puma, there is still a lot of things to change inside mastodon to make it a long term viable solution. And I'm not aware of any benchmarks tools done to test mastodon responsiveness (all I have personally is graphing tools for nginx and things like that)

[lanodan]

by the way pleroma moved to glitch-soc as an upstream for the mastodon frontend bundle and further stuff can be picked up to be more backend-indenpendent. (our modifications are done with pleroma in mind tho)

Git repository: https://git.pleroma.social/pleroma/mastofe