Delta Client Linux support tracking issue

[b-ncMN]

As a follow-up from #112 I am opening this issue to track the work to be achieved to have the client work on Linux

[stackotter]

Overview

• [x] Make all dependencies successfully build on Linux or find alternatives
• [x] Make Delta Core build on Linux (with rendering disabled)
  • By this I mean get it to a point where someone could write a bot client with it
• [ ] Write a renderer for Linux
• [ ] Write a UI for Linux
• [ ] Find a way to distribute the client for Linux

Dependencies

• [x] zipfoundation
  • Relies on Apple's libcompression so an alternative zip library would have to be used. This library was the most performant one I could find, so maybe only use a separate one on Linux and keep this one on Apple platforms (unzipping is one of the longest steps on first launch).
• [x] idzswiftcommoncrypto
  • This relies on Apple's CommonCrypto library
  • We might be able to move to this CommonCrypto port that adds Linux support
  • Or maybe the Perfect-Crypto package.
• [x] delta-logger
• [x] niodns
• [x] swift-protobuf
• [x] swift-collections
• [x] swift-concurrency
  • It seems like this package is only written with Apple platforms in mind. But it might build on Linux, just need to build it and see.
  • If it doesn't support Linux, it would probably be easy enough to find a different atomics solution (if I remember correctly, I only used this package in one file).
• [x] ecs
• [x] zippyjson
  • It seems like this package was only created for Apple platforms. But it might not be that far off being able to build on Linux. Either way, this package isn't essential and could just be replaced with the regular JSONDecoder on Linux until support is added, because this package's only purpose is to be a faster JSONDecoder.
• [x] swift-parsing
• [x] swift-argument-parser
• [x] swordrpc
  • This might be able to be built on Linux (assuming that Discord rich presence works the same way on Linux) because its only bluesocket which supports Linux.

Overall, it shouldn't be too much work to get all of the dependencies building on Linux.

Delta Core

I don't think there is too much platform specific code in Delta Core that isn't related to the UI or rendering. It should be relatively easy to use compiler directives to skip them on non-Apple platforms.

Rendering

Rendering is currently implemented using Apple's Metal framework. A renderer would have to be written for Linux using Vulcan or WebGPU or something. WebGPU would be preferable, but swift-webgpu would need to be made a bit easier to use. It would be good to abstract over the low level rendering stuff a bit so that mesh generation code can be the same for both renderers.

UI

The client's UI is currently made with SwiftUI which is closed-source and only supports Apple platforms. When making this Linux port we'll probably need to make a second UI that uses Gtk or some other cross-platform solution. My SwiftCrossUI package could be cool but it's pretty experimental at the moment and not ready for making any remotely complex UIs.

Distribution

Swift currently has issues producing statically linked executables. The best way to distribute Delta Client will probably be as a .deb for now (the most common dists that I know of use that). An alternative could be distributing it as a folder containing the libraries and stuff and then having a wrapper script to launch it with the correct PATH and library search path set.

Distribution isn't the top priority during development, so this can be left until a little further down the line.

[stackotter]

@InfRandomness made some progress on the distribution aspect.

Static binaries can be built using swift build -Xswiftc -static-stdlib -Xswiftc -static-executable. They also had to install libstdc++-static for this command to run successfully.

Binaries are huge, but do at least work.

[b-ncMN]

ZippyJSON Linux support : https://github.com/michaeleisel/ZippyJSON/issues/6

[stackotter]

IDZSwiftCommonCrypto has been replaced with OpenSSL (pending whether it builds on Linux, but it should). ZippyJSON is being disabled on Linux until it adds Linux support. SIMD was replaced with FirebladeMath (but will need some PRs in future to add all required operators).

Network.framework will likely be replaced by Socket.swift, because swift-nio looks overcomplicated for Delta Client's simple use case.

[ninjadev64]

All distros support AppImage, so if Swift supports that, then it would be a great option as well as the .deb for users on non-Debian based systems.

[stackotter]

Swift supports neither, that’s what Swift Bundler will be for, but I am definitely leaning towards only supporting AppImage. Cause its closer to how Mac apps work and it supports all distros. And a simple wrapper deb around an AppImage should be simple to create to support apt and stuff.

[ninjadev64]

I personally hate appimages because they don't "install" properly to app launchers without something like AppImageLauncher. So a deb would be nice too.

[Bixilon]

flatpak

[b-ncMN]

I disagree with all of those point and strongly argue we should ship a statically linked musl binary alongside some packages upstream, for those who are wishing to create and maintain them.

[stackotter]

I disagree with all of those point and strongly argue we should ship a statically linked musl binary

alongside some packages upstream, for those who are wishing to create and maintain them.

How does that work with the required resources and dynamic linking (for the plugin system)? The app kinda needs to be a few separate files, and I think AppImage can do that and combine them nicely into one? .deb would be the second easiest probably but that's annoyingly distro specific.

[ninjadev64]

134 introduces CustomJSONDecoder as a wrapper for JSON decoding, using ZippyJSON on Apple platforms and Foundation on Linux. Additionally, with help from @LeoDog896, the PR successfully replaces swift-concurrency with swift-atomics.

[ninjadev64]

Also worth noting that the only thing preventing SwordRPC being built on Linux was a typo (PKBeam/SwordRPC#1)

[stackotter]

OpenSSL has now replaced IDZSwiftCommonCrypto which brings Delta Core one step closer to building. I think the next step will be to separate rendering into a separate target so that Delta Core can build on Linux and be used to create bots etc until a Linux renderer is created.

[stackotter]

FirebladeMath has now replaced SIMD and everything seems to be working correctly on macOS still. I have multiple PRs in the works to improve the library but for now I'm just using a branch on my fork.

[stackotter]

I've replaced all of the networking with a custom Socket implementation now so we don't need to rely on Network.framework anymore (which is Apple platforms only). As an added bonus, we don't rely on NioDNS (and therefore NIO) anymore because before I was relying on Network.framework for most of the DNS resolution and NioDNS wasn't capable enough to do it all. So I switched to a much more standalone DNS library that is way nicer to use too (because no NIO).

Now I've run into Compression (another Apple platform only library) so I'll look into finding an alternative library for performing zlib compression/decompression.

[stackotter]

I've replaced Compression with a zlib library now and it seems to be working well

[stackotter]

I think all that remains now to get a working build is to split rendering code into a separate target so that DeltaCore can be fully built on Linux.

[stackotter]

I've split out rendering code and removed reliance on CoreGraphics now (by replacing image loading/processing with SwiftImage) and now I'm working on replacing CryptoKit (used for SHA1, RSA and shared secret generation.

[ninjadev64]

Thanks! I hope adding all these libraries for cross platform doesn't make build and runtimes too much slower.

[stackotter]

So far I think it has improved build times maybe cause we don't need to build stupid swift nio. But I'll definitely be benchmarking runtime performance before I merge, hopefully it won't be affected too much. I'll probably have to do some optimisation.

[ninjadev64]

Just thought I should update this thread - #184 and #185 implement server list UI and configuration persistence on Linux respectively.

[furby-tm]

👀

[furby-tm]

The linux branch is a year old, is this the branch I should be working out of -- or begin a new fresh branch based off inventory?

It was deemed the linux branch had already been merged into main a long time ago -- I have created a new branch based off inventory, and will attempt to usher in Linux support now.