Publish to major package managers

[theRookieCoder]

We would ideally want to publish to the most common package managers so that updating to the latest version is easy and convenient. These are the ones I have in mind;

• [x] GitHub Releases
• [x] Crates
• [x] AUR
• [x] Scoop
• [x] Nix Packages
• [x] Pacstall
• [x] Homebrew
• [ ] Chocolatey
• [ ] Debian or APT

Preferably, these packages should use the existing Releases builds rather than compiling separately. Their packages should also be updated automatically in the release Actions workflow.

[Southpaw1496]

Another update: I have found that Homebrew's core repository has a policy on "Niche or self-submitted stuff" which says that they will "reject formulae that seem too obscure, partly because they won’t get maintained and partly because we have to draw the line somewhere" so I think a custom tap may be something we actually need. Fortunately, setting up a custom tap is trivial, although providing custom bottles seems to be more complex. I'm also looking into how difficult it would be to automate, since that's something you said should be a thing.

[theRookieCoder]

I think we should try to use the GitHub Releases for the tap, and of course compile from source when passed as an option. Then there's no need to separately build bottles

[Southpaw1496]

From my limited initial research, bottles have some secret metadata sauce which makes them different from regular binaries.

I have found https://brew.sh/2020/11/18/homebrew-tap-with-bottles-uploaded-to-github-releases/ , but I haven't looked into that extensively yet.

Taps also seem to need to be specially formatted repositories

[theRookieCoder]

I have found https://brew.sh/2020/11/18/homebrew-tap-with-bottles-uploaded-to-github-releases/ , but I haven't looked into that extensively yet

I've read that many times, and it's the opposite of what we want. It builds bottles then uploads them to GitHub releases rather than the other way round.

[Southpaw1496]

I don't think we can get bottles from the GitHub Actions binaries.

[JustSimplyKyle]

This is another package of ferium, although not maintained by me. https://aur.archlinux.org/packages/ferium This is the stable release compiled from source. Which is different from -git, which is always the newest.

[theRookieCoder]

I emailed the maintainer, and requested deletion on AUR because it's unofficial and can be confused with the official packages

[Nusselt-Number]

Hello, I was looking to package ferium into a .deb for inclusion into the Debian and by extension Ubuntu repositories. I was using the instructions provided by the debian rust packaging team: https://salsa.debian.org/rust-team/debcargo-conf/-/tree/master. I originally tried to package version 4.1.0 using the tooling the Debian team provides, but got error messages asking about missing & too new dependencies. Debian doesn't package octocrab, online, dialoguer, ferinth, furse, or libium (I know those last 3 are sister projects so I would package them as well). I asked for advice in the debian rust packaging team's IRC and they suggested I try to write a patch to get ferium to build against older dependency versions. I tried to dive back in and give it another go today using ferium version 4.1.3 and get another error that the "strip" feature that isn't stable in the 1.56.0 verison of cargo (version packaged in Debian unstable). I'm not sure what the best step forward would be if our goal is to have a native .deb package included in the Debian/Ubuntu repos. I see a couple options from reading and googling, but I'm spitballing here:

1. "Cheat" and try to bundle all the dependencies in the deb, but this would all-but eliminate the ability to get the package into the official repositories. Also I don't know what this really gets us over the pre-built binaries in the releases page.
2. Wait for Debian to package newer versions of the dependencies and submit missing packages for inclusion. Not sure how long this would take and we'd need a sponsor to get the packages into the repo because I'm not a Debian Maintainer or Debian Developer.
3. Give up on a native package for Debian and just advise people to download a binary from releases or from Cargo.

I gave an attempt at packaging ferium for Fedora's COPR which provides rpms for RHEL derivatives (CentOS, Rocky, Alma), Fedora, etc. I think there are similar problems that we'd have to address if we want official packages.
Rust packaging documentation for Debian: https://wiki.debian.org/Teams/RustPackaging, Fedora: https://docs.fedoraproject.org/en-US/packaging-guidelines/Rust/. I apologize I don't have much in the way of solutions or answers. I'm not a software developer, just an end-user.

[Blackcatmaxy]

I gave an attempt at packaging ferium for Fedora's COPR which provides rpms for RHEL derivatives (CentOS, Rocky, Alma), Fedora, etc. I think there are similar problems that we'd have to address if we want official packages. Rust packaging documentation for Debian: https://wiki.debian.org/Teams/RustPackaging, Fedora: https://docs.fedoraproject.org/en-US/packaging-guidelines/Rust/. I apologize I don't have much in the way of solutions or answers. I'm not a software developer, just an end-user.

I got this building on OBS for OpenSUSE (and Fedora for a while) although I ran into a similar issue with Rust 1.61 not being available even on Rawhide Fedora after an update to Ferium.

[theRookieCoder]

Rust 1.61 not being available even on Rawhide Fedora after an update to Ferium

I'm also having the same issue with Homebrew. Unfortunately that 1.61 feature is pretty useful because it allows destructors to properly run before exiting

[theRookieCoder]

got error messages asking about missing & too new dependencies

Was this a problem with the Rust code not compiling? If so, this is related to #113, and I've fixed this in 80cb7e39e2e19f7d8db87577255603f5ca3bfe18

[Nusselt-Number]

No, not a compile-time error that I was aware of. The helper scripts the Debian rust packaging team provides said that certain dependencies (octocrab, dialoguer, libium) didn't exist as packaged rust debs. I ran the cargo-debstatus crate to check dependencies that were packaged in Debian, output here: https://paste.debian.net/1243842/.

[Southpaw1496]

Rust does now have a Ruby file ready to go on Homebrew, but it hasn't been merged yet.

[theRookieCoder]

certain dependencies (octocrab, dialoguer, libium) didn't exist as packaged rust debs

But those are Rust dependencies from crates.io, they will be compiled together into the executable. Why should a package manager care about those?

[ImperatorStorm]

My OBS packages use debbuild to build .debs from RPM .specs, though this would absolutely not be accepted into official repos.

[Nusselt-Number]

Why should a package manager care about those?

Because if you want a .deb package that will be accepted into official repos you can't just ship an executable with everything already bundled together. It has to be able to be built locally from all the source tarballs of crates without accessing the internet (i.e. crates.io). This means all the crates it depends on have to be packaged and exist in the Debian repos. From the Debian Rust Packaging Policy:

Package builds must not allow Cargo to access the network when building. In particular, they must not download or check out any sources at build time. Instead, builds must use the packaged versions of crate sources, via the corresponding library crate packages, which provide a Cargo directory registry.

[theRookieCoder]

Huh, that's extremely weird and annoying to say the least. So how do people upload their Rust packages to debian? Is this why I've never seen any of my Rust tools in apt?

[theRookieCoder]

I found this from the Rust CLI book, I wonder if we can just use this

[Nusselt-Number]

So how do people upload their Rust packages to debian? Is this why I've never seen any of my Rust tools in apt?

Yeah, I would imagine, it can definitely be a PITA. Policy is also similar on most RHEL derivatives to my knowledge, Fedora's policy for example states the following:

Fedora packages SHOULD make every effort to avoid having multiple, separate, upstream projects bundled together in a single package. All packages whose upstreams allow them to be built against system libraries MUST be built against system libraries. In this case, bundled libraries (and/or their source code) MUST be explicitly deleted during %prep.

Fedora also has specific rust packaging guidelines and provides tooling for packaging crates, much like Debian: https://docs.fedoraproject.org/en-US/packaging-guidelines/Rust/

If you're curious, you can clone the Debian Rust Packaging team's git here: https://salsa.debian.org/rust-team/debcargo-conf and look through the source folder to see which rust crates they currently package.

[theRookieCoder]

https://github.com/theRookieCoder/homebrew-tap

I created a simple tap formula that uses GitHub Releases. No compiling from source or support for platforms other than macOS x64 yet

[Nusselt-Number]

I found this from the Rust CLI book, I wonder if we can just use this

Good find, I gave it a try and after installing some development packages, I am able to get a binary .deb package after running cargo deb. This package won't adhere to Debian policy, so it most likely won't be allowed in official repos. But if our goal is to just have something good enough for the Github Releases page that people can install with sudo dpkg -i, this should work.

[Elsie19]

Hey. I'm adding Ferium to Pacstall (https://github.com/pacstall/pacstall-programs/pull/1368), and I noticed you don't have a repology card on your README. Would you be willing to add Pacstall as a method of installing Ferium on your README? I can make the PR if need be.

[theRookieCoder]

I'll look into adding the repology card. If your the pacstall PR gets merged I'll be more than happy to add it to the readme!

[Elsie19]

Alright we merged it!

[theRookieCoder]

homebrew/homebrew-core#106034

[theRookieCoder]

@Southpaw1496 what do you think?

[theRookieCoder]

Yet another package manager, ferium is now available on Homebrew!

[JustSimplyKyle]

https://aur.archlinux.org/pkgbase/ferium-git/ Does anyone have any idea why it can't build in clean chroot?

[theRookieCoder]

I think the issue is that the zip crate depends on bzip2 and Zstandard. (zstd is what's missing in your build failure) Actually those are used for compression, but ferium only needs to decompress modpacks (at least till #98 is implemented). So I should be able to just disabled those features, then neither bzip2 nor zstd would be required!

[theRookieCoder]

Damn like 20 dependencies just went poof. Maybe I should go through all the crates and disable unnecessary features...

[theRookieCoder]

Oh yeah @JustSimplyKyle both bzip2 and zstd were removed so those are no longer required

[theRookieCoder]

@JustSimplyKyle Do the binaries still have a dependency on bzip2? If not, the AUR packages seem to still specify it as a dependency, could you remove that too?

[JustSimplyKyle]

No problem! This kinda flew over iy head, will do!

[theRookieCoder]

@imsofi is there a way to automatically create a PR that updates the nixpkgs ferium package? For example for homebrew I was able to find https://github.com/mislav/bump-homebrew-formula-action but I just can't find anything like that for nixpkgs.

[soupglasses]

@r-ryantm is a bot in nixpkgs that auto-updates packages. For example: https://github.com/NixOS/nixpkgs/pull/210427#event-8216431715

They use the https://github.com/ryantm/nixpkgs-update project to update, but its more designed to update the entirety of nixpkgs.

https://github.com/Mic92/nix-update could be closer to what you are looking for. Should be runnable with something like nix-update ferium

[KokaKiwi]

@theRookieCoder

I emailed the maintainer, and requested deletion on AUR because it's unofficial and can be confused with the official packages

Hello, I just re-instated my (updated) AUR stable source-based package for ferium at: https://aur.archlinux.org/packages/ferium

It would be nice to simply flag as out-of-date the package (there's a dedicated link for that on AUR package pages) instead of requesting their deletion, especially when i already replied why i created this AUR package the last time you requested its deletion (and was at the time rejected because of the lack of other stable source-based AUR package)

[theRookieCoder]

Uhh I did not get any message/email from you about your reasons for this package existing, my Discord is theRookieCoder#1287 if you'd like to chat. There are source-based packages available (those suffixed with -git) btw so your package still qualifies as a duplicate

[theRookieCoder]

was at the time rejected because of the lack of other stable source-based AUR package

The packages back then were exactly the same as they are right now, I just did not properly link to them in my request.

[KokaKiwi]

Uhh I did not get any message/email from you about your reasons for this package existing, my Discord is theRookieCoder#1287 if you'd like to chat

I replied to the deletion request email i had, but not the other one you sent (i think it went lost among the spam i receive, sorry ><), so maybe that's why uh :(

In my reply i actually proposed to add you as co-maintainer of the package so you would be able to update the package yourself when needed

There are source-based packages available (those suffixed with -git) btw so your package still qualifies as a duplicate

There's a huge difference between "latest commit source-based" packages and "stable source-based" packages actually. The AUR has basically this naming-scheme for packages:

• <package> : Package which build from source against the latest stable release
• <package>-git : Package which build from source against the latest commit on the repository, often considered as "unstable packages", since it's akin to a "dev build"
• <package>-bin : Package which build from binary release against the latest stable release, generally used on platform not able to compile stuff (for instance i use them on my rPi ^^)

[theRookieCoder]

I replied to the deletion request email i had

hmm I didn't receive anything, I checked my spam/junk inboxes but I can't find anything. Anyways,

There's a huge difference between "latest commit source-based" packages and "stable source-based" packages actually.

Thanks for the clarity! (and apologies for the confusion 😬)
Does the 'stable' source-based use the latest git tag?

Well there is some missing information in your package:

• There is no GUI version of the package
• GCC is not required since ferium uses musl to build
• Not sure if this is strictly necessary but you should probably add git as a dependency too
• The repository url hasn't had the organisation name updated
• You should fill out the conflicts and provides fields (particularly because there should ideally be 6 packages for ferium, which would all conflict)

One last doubt, does this package need manual updating of the git tag? That can be added to the workflow so it updates automatically whenever I push and update. If you need any help, I'm pretty sure @JustSimplyKyle can give you hand 😁

[KokaKiwi]

Thanks for the clarity! (and apologies for the confusion :grimacing:)

It's okay, i understand how confusing AUR can be for people ^^

Does the 'stable' source-based use the latest git tag?

Yup, i actually enabled notifications for this repository when releases are pushed so i could update my package :smile: (but seems i actually forgot to do this since some times uh ><)

• There is no GUI version of the package

No worry, in the time you wrote this i actually pushed the GUI version of my package: https://aur.archlinux.org/packages/ferium-gui :p

• GCC is not required since ferium uses musl to build

Actually for source-based packages, you need gcc-libs as dependency since on arch it builds by default against gcc, not musl (unless i specify the --target manually which can be tricky when using the rust package from official repositories)

• Not sure if this is strictly necessary but you should probably add git as a dependency too

The source the package use is actually the archive version of the repository, which allows using checksums for source-integrity checking, so git is not necessary :)

• The repository url hasn't had the organisation name updated

Whoops i didn't notice so i'll update this ASAP ^^

You should fill out the conflicts and provides fields (particularly because there should ideally be 6 packages for ferium, which would all conflict)

Actually that shouldn't be needed since the other packages already have the provides field, which automatically setup the conflicts rules and stuff Also adding provides=('ferium') on a package already named ferium is basically a noop so it's not needed either :)

One last doubt, does this package need manual updating of the git tag? That can be added to the workflow so it updates automatically whenever I push and update. If you need any help, I'm pretty sure @/JustSimplyKyle can give you hand :grin:

As i said, i'm actually notified on every new releases you push here so i can update my AUR packages when needed (i do that for every packages i maintain on the AUR ^^) It seems i didn't took much time to update some packages (mostly because of life stuff ><), so i forgot to update the package :x But on those cases don't hesitate to push the "Flag as out-of-date" link on the AUR package page so it'd yell at me to update the package xD

idk about how we could hook the workflow to update the package but i'd be interested for help on how to do that yeah :D

[JustSimplyKyle]

Actually that shouldn't be needed since the other packages already have the provides field, which automatically setup the conflicts rules and stuff

Oh really, that's cool! So I can just remove every conflict that I have?

Actually for source-based packages, you need gcc-libs as dependency since on arch it builds by default against gcc, not musl (unless i specify the --target manually which can be tricky when using the rust package from official repository

I've tried to build my packages in clean chroot, and it does build!(using musl)

[JustSimplyKyle]

To use the workflow, first add Ilesh to co-maintainer Then it should be as simple as adding the following to the release.yml

      - name: Update gui AUR package
        uses: ATiltedTree/create-aur-release@v1
        with:
          package_name: ferium-gui
          commit_username: "Ilesh Thiada"
          commit_email: ileshkt@gmail.com
          ssh_private_key: ${{ secrets.AUR_SSH_PRIVATE_KEY }}

[Nukecraft5419]

Could we also have a version for npm, and also a discord to have better support and contact also

[theRookieCoder]

Hmm not sure why ferium would need an npm package? It has absolutely nothing to do JavaScript.

I never really expected this project (or any of my projects) to go this far, so I haven't considered a Discord server yet. You can use the GDLauncher server for now in the gdlauncher support channel with the Ferium tag.

[Nukecraft5419]

Basically I'm creating a hybrid Launcher between the official Minecraft Launcher and MultiMC based on Electron and uses javascript/typescript I'm still deciding on this thing because there are 2 versions of electron then I'll use React and Chakra UI and I also have a client similar to Lunar and others I'm all about fabric with all mods not made by me and I wanted to make that only in my launcher there was the download via the ferium script

[RokeJulianLockhart]

@theRookieCoder, you'll get a lot more exposure if you manage to publish to

0. [ ] snap and
1. [ ] flatpak (at least for the GUI version)

since they support every Linux distribution except Android. Additionally, flatpak is the sole method of persistent installation of Fedora (Silverblue and Kinoite).

[theRookieCoder]

I'm not all that familiar with Linux but aren't these for GUI desktop applications? Ferium is still CLI only and there aren't any plans for a GUI port yet.

But yeah your concern is valid, I find it frustrating too that there isn't an easy way to reach all (if not most) Linux users easily. I'm thinking of using Fedora and it pains me to not see my own software on it.

[ImperatorStorm]

Snap works for both cli and gui, but is jank in other ways. Flatpak can work for CLI, but iirc it'll take additional user input

[soupglasses]

But yeah your concern is valid, I find it frustrating too that there isn't an easy way to reach all (if not most) Linux users easily. I'm thinking of using Fedora and it pains me to not see my own software on it.

One universal packaging method available to all distros right now is nix. Which on Fedora you can install trough a dnf package from here: https://github.com/nix-community/nix-installers

There is also the official method, which works well on other distros if not supported by nix-installers: https://nixos.org/download.html

Then running nix profile install nixpkgs#ferium should put it into your PATH :+1: