Publish to major package managers

[theRookieCoder]

We would ideally want to publish to the most common package managers so that updating to the latest version is easy and convenient. These are the ones I have in mind;

• [x] GitHub Releases
• [x] Crates
• [x] AUR
• [x] Scoop
• [x] Nix Packages
• [x] Pacstall
• [x] Homebrew
• [ ] Chocolatey
• [ ] Debian or APT

Preferably, these packages should use the existing Releases builds rather than compiling separately. Their packages should also be updated automatically in the release Actions workflow.

[NatoBoram]

There's a good alternative to Chocolatey on Windows: https://scoop.sh

It actually manages packages instead of just running the installer, like choco and winget.

[JustSimplyKyle]

yeah, scoop is great too. Although chocolatey is the most well known(scoop is just behind)

[theRookieCoder]

It seems that Scoop is meant more for developer tools? Well it seems much easier to upload to anyways so I'll do both

[JustSimplyKyle]

https://gist.github.com/KyleUltimate/3e20b14ecbf6a5512912f8dc3addc4b7 This is my implementation of ferium in scoop. It's really easy to implement. This should not even require action changing, the auto update should be done by scoop (currently there is no hash checking because I'm lazy to check hash for windows)

[JustSimplyKyle]

I will add it later, it should also automatically update sha256sums too.

[JustSimplyKyle]

Ok now it also auto updates the hash too!

[JustSimplyKyle]

Also, do you consider ferium fitting this? https://github.com/ScoopInstaller/Scoop/wiki/Criteria-for-including-apps-in-the-main-bucket I think it does. Therefore https://github.com/ScoopInstaller/Main/pull/3558

[theRookieCoder]

Also, do you consider ferium fitting this?

I wasn't actually sure, I mean it isn't 'reasonably well-known' or a 'developer tool' right?

[theRookieCoder]

Oh and looking at the autoupdate, would the releases now need .sha256 files? Or is that some sort of shortcut?

[JustSimplyKyle]

I believe it's a shortcut! Although I'm not sure bout that

[JustSimplyKyle]

Wait hmm no. It's an direct file. We still need to upload the sha256sum to github release too

[JustSimplyKyle]

https://github.com/theRookieCoder/ferium/pull/50 Done, haven't actually tested that tho. It's a very simple one, it should just work

[JustSimplyKyle]

https://github.com/Calinou/scoop-games/pull/616 Up!(This is the official game repository) Installation step.

1. scoop bucket add games
2. scoop install ferium And it also auto upgrades! My pull request is 3.27.0, but it auto upgrades to 3.28.0!

[theRookieCoder]

This is great, thank you so much! Also where did you get the screenshot from, I can't find a Scoop index website with Ferium in it

[JustSimplyKyle]

https://scoop.sh/#/apps?q=ferium

[ImperatorStorm]

You can create packages for multiple distros(Debian+Debian-based, Arch, basically all of the RPM distros...) and automatically make repositories for them via https://build.opensuse.org.

You can use https://build.opensuse.org/package/show/home:ImperatorStorm/minizip-git as an example if needed, and via a beta automatically trigger an OBS rebuild on commit.

[JustSimplyKyle]

Another AUR release! https://aur.archlinux.org/packages/ferium-git This is ferium-git, which automatically compiles from source! The package version will always be "outdated", but the actual package won't... In short words, it auto upgrades and doesn't require any changes to ferium itself

[theRookieCoder]

How does it know when it should recompile, does it infer that from the tags/commits?

[JustSimplyKyle]

Well the thing with git packages is that the upgrading is done on the user side. When the user decides to build(or upgrades) the package, it automatically pulls from the latest commit. But the upstream package version will never get upgraded. eg. The left version continuous.r1621.g41a49c488-1 is what you see on AUR But the right Installed: continuous.r1940.g6613ff329-1 is the version you actually get

[theRookieCoder]

Ah I see, and now from-source builds don't need the CurseForge API key environment variable anymore so perfect timing!

[JustSimplyKyle]

Actually, you don't need to state that it require the rust toolchain since it has cargo in makedepends(which automatically configures the rust toolchain for you, and you can choose to uninstall makedepends after your installation but that's normally not recommended although it would function)

[theRookieCoder]

I just wanted to warn the user that it will install additional stuff that might be quite large because the Rust toolchain is over 5gb I think

[JustSimplyKyle]

hmm, I can't get just build-linux to work, but I could get just build-linux-nogui to build? it seems like build-linux will also build for windows but I don't have the windows toolchain so it doesn't work. Currently I'm tweeking the -git package to automatically build the nogui version if you don't have gtk3 installed, and build the normal one if you do have one installed. But I'm running into this problem... Is this the same problem that this issue has? https://github.com/theRookieCoder/ferium/issues/69

[ImperatorStorm]

Looks like it, cargo doesn't auto-install toolchains that aren't for the host OS, you'd need to install them manually. Fixed in https://github.com/theRookieCoder/ferium/pull/71 once it gets merged.

[JustSimplyKyle]

https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=ferium-git Yep after the latest commit, this now works!

[ImperatorStorm]

why keep it as a single package, rather than split it like the binary packages?

[JustSimplyKyle]

Because that's what people normally do for source packages, enable features if your computer has that functionality. But for binary packages, it's not the normal thing to do Also another reason, I don't know how to download only a single source :rofl:

[ImperatorStorm]

Nah, people split packages into cli and gui versions commonly, see citra-git and dolphin-emu-git. Also, changing build depending on whether a certain package is available breaks reproducible builds.

Rewrote the ferium-git PKGBUILD: https://gist.github.com/ImperatorStorm/bbe40ed329f5706740b54c4e9bec0933

Unrelated, but

gtk optdepends should be gtk3, as gtk refers to a gtk1 aur package (https://aur.archlinux.org/packages/gtk)

[JustSimplyKyle]

oh, my apologies. https://aur.archlinux.org/packages/ferium-git https://aur.archlinux.org/packages/ferium-gui-git And the ferium-gui-bin dependencies is set to gtk3! Thanks so much for your help, I still need to learn a lot about AUR packaging!

[leo60228]

I opened a PR for NixOS/Nixpkgs: https://github.com/NixOS/nixpkgs/pull/173201

[Blackcatmaxy]

You can create packages for multiple distros(Debian+Debian-based, Arch, basically all of the RPM distros...) and automatically make repositories for them via https://build.opensuse.org.

You can use https://build.opensuse.org/package/show/home:ImperatorStorm/minizip-git as an example if needed, and via a beta automatically trigger an OBS rebuild on commit.

I have created a working package for OpenSUSE Tumbleweed on the OBS, I recommend branching my project and integrating it into the workflow with the guide mentioned above for autobuilds. I can also see if I can enable support for other RPM based distros if there is any interest.

[theRookieCoder]

I opened a PR for NixOS/Nixpkgs: NixOS/nixpkgs#173201

@leo60228 I've released v3.28.8 so that patch can be removed. You should probably add the macOS Intel and arm assets too.

[theRookieCoder]

I have created a working package for OpenSUSE Tumbleweed on the OBS, I recommend branching my project and integrating it into the workflow with the guide mentioned above for autobuilds. I can also see if I can enable support for other RPM based distros if there is any interest.

I'm quite confused as to how to integrate this into the workflow. I'm not very familiar OpenSUSE

[ImperatorStorm]

I'm quite confused as to how to integrate this into the workflow. I'm not very familiar OpenSUSE

See https://openbuildservice.org/help/manuals/obs-user-guide/cha.obs.scm_ci_workflow_integration.html

I can also see if I can enable support for other RPM based distros if there is any interest.

Just enable them and see which ones break.

Also, you can enable debian-based support with a project config like this. Use a subproject specifically for Debian-based distros like I did here so you don't break the builds for non-Debian-based distros.

Also, enable the services in https://build.opensuse.org/package/view_file/home:Blackcatmaxy/ferium/_service?expand=1.

[Blackcatmaxy]

Also, enable the services in https://build.opensuse.org/package/view_file/home:Blackcatmaxy/ferium/_service?expand=1.

Tried that and it seemed to break everything, had to revert that change. For the record I'm trying to follow these docs although I already figured out that it led me astray once by having auto updating cargo enabled in the services.

[ImperatorStorm]

Tried that and it seemed to break everything, had to revert that change. For the record I'm trying to follow these docs although I already figured out that it led me astray once by having auto updating cargo enabled in the services.

Submitted a request to fix the auto-download service, should just download on version bump. You will need to bump the version in both the _service file and the .spec file, however. Also, this should work fine for most rpm distros.

[ImperatorStorm]

Fedora 36 doesn't seem to package cargo-packaging, which Fedora 36 needs to build Ferium. You're probably going to need to branch it to your home:Blackcatmaxy project, build it for Fedora 36, and then trigger a rebuild for Ferium.

[Blackcatmaxy]

Now getting unresolvable: have choice for (glibc-langpack-en or glibc-all-langpacks) needed by obs-service-obs_scm-common: glibc-all-langpacks glibc-langpack-en. Figure I want to do something like "if Fedora buildrequire glibc-langpack-en"?

EDIT: Did that and got it working (although I'm not going to test it, at least not anytime soon maybe will get a VM later), think there's any point in the other architectures?

[ImperatorStorm]

Can probs add i386 and aarch64 support, iirc i386 is supported <1.19 and aarch64 linux natives are bundled in lwjgl in 1.19+. Don't remember the exact snapshots, so I could be incorrect.

[leo60228]

You should probably add the macOS Intel and arm assets too.

@theRookieCoder I don't have a Mac to test so that'll be a bit annoying. I added Security.framework, which seemed to be the only issue on CI?

[theRookieCoder]

Hmm what do you need to check on macOS, I might be able to help you. But again I really don't understand how nixpkg and obs work

[Blackcatmaxy]

Hmm what do you need to check on macOS, I might be able to help you. But again I really don't understand how nixpkg and obs work

OBS is essentially a giant CI/CD service (think an open source GitHub actions but oriented around packaging and even bigger) for building traditional applications like .rpms, .debs, and .appimages. (EDIT, also a key part is allowing users to create package update repositories in a secure and transparent manner). NixPkgs (probably not going to do them justice) are closer to Flatpaks in that they're a newish package format that expresses their dependencies and can actually be installed on any distro (and even on MacOS?), it's something I've been meaning to try out but haven't had time.
These are both oversimplifications and I recommend reading some of the links included although the Nix docs are a bit overwhelming.

[soupglasses]

I opened a PR for NixOS/Nixpkgs: NixOS/nixpkgs#173201

Ah you seem to have gotten to it before me, I am currently running a NUR package for ferium here: https://github.com/imsofi/nur-pkgs/tree/main/pkgs/ferium

[soupglasses]

Hmm what do you need to check on macOS, I might be able to help you. But again I really don't understand how nixpkg and obs work

Nix is just a package manager, in a similar vein to apt, dnf and homebrew. But its unique feature is that every step to build a package (called a derivation under nix) is completely versioned, similar to how git does versioned commits.

For how to help, you can take a look at installing nix (the package manager) trough https://nixos.org/download.html#nix-install-macos . There is likely other alternatives around which dont use the sh < (curl url) antipattern, but its the simplest way i know get it installed properly on macos.

For testing, you could clone my repo at https://github.com/imsofi/nur-pkgs/ . It is much lighter than attempting to pull in the entirety of nixpkgs, as its just my own personal NUR repo (NUR meaning Nix User Repository, think of the AUR if you know about Arch Linux).

And once you are at the root of that git repo. You can run nix-build -A ferium to build the ferium package trough nix. If it builds correctly, it should give you a ./result folder with ./result/bin/ferium which is the compiled rust package.

You will probably hit issues with missing dependencies. I see that @leo60228 has added buildInputs = lib.optionals stdenv.isDarwin [ darwin.Security ]; to the ferium/default.nix file. You can try to edit the ./pkgs/ferium/default.nix file with that line to see if it helps you to build and run it properly. I am however having a hunch it might make a differently compiled file, which would break the cargosha256 part. You could replace the hash there with the hash you get from the nix-build output around got: sha256-xxxxxx= at the end.

Hope that can help.

[Southpaw1496]

I'm in the final stages of working on a Homebrew formula. I've got it to successfully build and run, and the last steps are to create some tests for it and audit it.

If you'd like me to, I'm happy to maintain the Homebrew package on your behalf, at least for the time being.

[Southpaw1496]

Update:

When I started the project, Ferium was on 3.96, and that built and ran fine, but during testing it was failing to parse the Modrinth API properly (error decoding response body: unknown fieldsize, expected one ofhashes,url,filename,primaryat line 1 column 795), so I tried updating it. On building the updated version, I received the following error:

error[E0658]: use of unstable library feature 'process_exitcode_placeholder'
  --> src/main.rs:15:11
   |
15 | use std::{process::ExitCode, sync::Arc};
   |           ^^^^^^^^^^^^^^^^^
   |
   = note: see issue #48711 <https://github.com/rust-lang/rust/issues/48711> for more information

After some research, I'm presuming that this is because Rust is on 1.61.0, whereas Homebrew is still on 1.59.0, due to some issues Homebrew seems to be having with updating the package (see https://github.com/Homebrew/homebrew-core/pull/98823). Unless I'm wrong about why the error occurred, we'll have to wait for them before Ferium can be on Homebrew.

[theRookieCoder]

I'm presuming that this is because Rust is on 1.61

Yes and Ferium uses the latest features

Homebrew is still on 1.59.0

Damn that sucks, yeah we'll have to wait for them then

[theRookieCoder]

Also is it possible to install the GitHub Releases binaries by default, and only compile from source if -s is enabled?

[Southpaw1496]

As far as I can tell, once you submit the file to the core repository, Homebrew's CI will build binaries ("bottles") for you, you don't provide them yourself.

We might be able to get around that by starting our own tap (something I haven't looked into because it wouldn't be necessary under normal circumstances), but that would be completely overkill since we can just wait.

It seems the issue is that the compiler fails if Rust is built without Jemalloc (https://github.com/rust-lang/rust/issues/92173) but adding Jemalloc pushes the build time to over three days when using GitHub actions, which exceeds some sort of limit, and they're discussing ways to optimise it at the moment.

[theRookieCoder]

pushes the build time to over three days when using GitHub actions

Oh damn!

I wonder, how long would compiling Rust and it's dependencies 100% from source take on my potato laptop? A week?