search

varabyte / kobweb-cli

The CLI binary that drives the interactive Kobweb experience. See varabyte/kobweb
Apache License 2.0
10 stars 5 forks source link

Epic bug: Please support my favorite package manager! #11

Open bitspittle opened 2 years ago

bitspittle commented 2 years ago

In order to avoid everyone creating one off bugs for the same package manager, this bug will act as a home to collect all such requests received so far. (Of course, once someone starts working on adding support for one of them, that could be a good time to create an individual bug).

Here is the current list of package managers we are considering (but not actively working on):

NOTE: Some of these package managers may never get supported, as I'm not sure how many people actually want to use them, and supporting more releases results in more work on our end.

So even if you see your package manager on this list, it still may be worth leaving a comment below like "Yes, I want that one please! I use it particularly because..."

If you see a comment below from someone expressing support for a package manager that you also want, please give their comment a thumbs up so we can better roughly understand what people are using.

russellbanks commented 2 years ago

It would be nice to have this on Winget! Currently however, Winget doesn't accept zip files but rather installers so we'd have to either wait until they do support zip files (it's an open issue) or create an executable (exe or msi).

bitspittle commented 2 years ago

Thank you @russellbanks for your comment. I updated the initial list to include winget.

Hmmm, it might be possible to create an exe... but at that point I think I would be bundling a jre which might balloon the final size a bit, which may be unfortunate since to use Kobweb the user pretty has to have Java installed on their machine already.

Hopefully, though, winget decides to support .zip files. It is so convenient and would reduce so much friction!

russellbanks commented 2 years ago

I'm not the most well-versed in packaging JVM applications (in terms of bundling the JRE, etc) but is it possible to create an executable without bundling the JRE? If this is not feasible, we could just bite the bullet and bundle the JRE with it, specifying that the recommended option is still the zip file?

Even if this is a temporary solution, I still think I would be advantageous to look into ways we can get an executable working in order to get this on Winget as it is the official package manager for Windows.

P.S. Just a correction to the updated initial list - the link for Winget is https://github.com/microsoft/winget-cli.

bitspittle commented 2 years ago

@russellbanks Thanks for letting me know about the wrong link. I fixed the original comment.

Even if we could build an .exe, just keep in mind I'd want a flow where this could all be done automatically every time I released a new version. With existing package managers, this is basically a one click operation for all of them so far. So we'd need to figure out how to automatically 1) unzip the file, 2) bundle it up into an exe, 3) and publish this to winget.

I personally won't have time to look into this for quite a while, but if someone knows how to do it, and/or has an example of this happening already, please add a comment -- that could help a lot!

P.S. Russell, you may want to check out the https://github.com/jreleaser/jreleaser project and file an issue with them to support winget. I'm relying on them pretty heavily for my other releases, and if winget is the official package manager for windows, they may be interested in getting some signal that people would want it supported.

russellbanks commented 2 years ago

Not much more I can comment on this other than I hope that either Winget supports zip files in the future or JReleaser adds support for Winget. There is an open issue on JReleaser already to ask for support for winget as a packager, but it doesn't seem to have any activity.

bitspittle commented 2 years ago

Great, thanks for the update! With JReleaser leading the way, I will definitely take a look.

On Fri, 6 May 2022 at 10:12, Begula @.***> wrote:

JReleaser now has an MSI installer which can be published to WinGet. I think we can now proceed.

— Reply to this email directly, view it on GitHub https://github.com/varabyte/kobweb-cli/issues/11, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKNONAWHPPHGI6JK3EKE66TVIVHINANCNFSM5NJWQPUQ . You are receiving this because you authored the thread.Message ID: @.***>

russellbanks commented 2 years ago

As of winget 1.4, there will be support for installing zip files. However, this is just a preview at the moment, with the stable version being 1.3. But, this means that Kobweb can be put on winget; it's just a matter of time.

aalmiray commented 2 years ago

FWIW you can use jpackage (on its own or via JReleaser) to create Windows installers (exe, msi) that may be used in combination with Winget.

bitspittle commented 2 years ago

(Sorry for the recent radio silence, folks. I just moved, which wiped out my schedule for a couple of weeks).

As I catch up on stuff, I do intend to look into Winget, although I have a few other priorities to get through first. I'm hoping to get to this by end of September ('22) at the very latest, so feel free to ping this bug if it's later with no progress.

bitspittle commented 1 year ago

Finally, I'm back to this bug and looking into adding winget support. Thanks again for everyone's patience.

@aalmiray I'm trying to follow the Gradle syntax for jpackage from here: https://jreleaser.org/guide/latest/reference/assemble/jpackage.html

but "jpackage" contents aren't autocompleting for me: jreleaser-jpackage

(I'm using latest, Jreleaser v1.5.0)

I'm using kotlin script BTW. Maybe "app" is some dynamic thing that works only in groovy scripts?

Is there any demo code for using jreleaser + jpackage I can look at?

Also, according to @russellbanks , winget should support .zip files now (1.4 was released in Jan). Are there plans to add support for that into JReleaser, or will jpackageing be the way to go for a while?

russellbanks commented 1 year ago

@bitspittle

https://github.com/jreleaser/jreleaser/issues/185

https://jreleaser.org/guide/latest/reference/packagers/winget.html#_distribution_support

bitspittle commented 1 year ago

Right, that's what I'm looking at. winget doesn't support .zip file targets according to that distribution support list, so I assume at this point I still need to try running jpackage to create an .exe for now.

aalmiray commented 1 year ago

Zip support in Winget was added in 1.4 but the team has decided to wait until a critical mass of users has migrated to 1.4. See https://github.com/jreleaser/jreleaser/issues/185#issuecomment-1406908541

Once that happens we'll support releasing zips with Winget.

re: auto-completion. app is a free name which you can choose to be anythign you want. It identifies a named element inside a NamedContainer, thus you must use regular KotlinDSL such as

jreleaser {
  assemble {
    jpackage {
      create("kobweb-installer") {
        ...
      }
    }
  }
}
bitspittle commented 1 year ago

Ah perfect, it was a Kotlin script thing. Thanks for the quick reply!

russellbanks commented 1 year ago

Zip support in Winget was added in 1.4 but the team has decided to wait until a critical mass of users has migrated to 1.4. See jreleaser/jreleaser#185 (comment)

@aalmiray zip's were turned on in the pipelines a month or so ago: https://github.com/microsoft/winget-pkgs/issues/87300

aalmiray commented 1 year ago

@russellbanks gotcha. Will have a look at zips on Winget in the next couple of weeks.

aalmiray commented 1 year ago

Filed a new ticket to keep track of zip support on Winget https://github.com/jreleaser/jreleaser/issues/1253

bitspittle commented 1 year ago

I think I'll hold off on winget until zip support is ready.

It looks like supporting building an MSI/EXE installer will be rather tricky, involving a bunch of structure just for winget that no other package manager has needed. For example, to run jlink / jpackage, I think I would need to set up a special CI workflow using Java 16, since everything else I've done works fine with JDK11 (or older).

And given the extra complexity around this, I have other priorities I should attend to first.

russellbanks commented 1 year ago

I can manually submit kobweb to winget for the time being if you'd like @bitspittle

aalmiray commented 1 year ago

Understood. If it helps at a later stage, have a look a JReleaser's release workflow and configuration which take care of multiple targets during a release.

bitspittle commented 1 year ago

Thanks @russellbanks , I appreciate the offer!

However, I'd want to make sure that everything is handled in the build script so whenever I update the Kobweb binary, I can do everything with a single jreleaserPublish.

Probably best to figure everything out the correct way all at once (or otherwise I might release future versions of Kobweb and winget would fall behind)

russellbanks commented 1 year ago

Anything you do do to automate winget should take the previous manifest and update the values with that of your new artifacts, so a pre-existing manifest shouldn't stop you doing that in the future.

bitspittle commented 1 year ago

@aalmiray Yeah I was using one of your simple workflows as a guide, but seeing that I'd need to set up a bunch of runtime JDKs just for winget seems like a lot of work.

Also, I can just jlink / jpackage a windows binary right? If I don't care about the other platforms?

And if I am building an installer for windows, do I need to make sure to run jreleaser on a Windows machine?

bitspittle commented 1 year ago

@russellbanks where does the winget manifest live? Currently, I have some local repositories (e.g. for homebrew) which jreleaser overwrites for me. Is winget different? (I was skimming the docs but haven't looked too closely yet)

aalmiray commented 1 year ago

Also, I can just jlink / jpackage a windows binary right? If I don't care about the other platforms?

Correct.

And if I am building an installer for windows, do I need to make sure to run jreleaser on a Windows machine?

Correct, again.

russellbanks commented 1 year ago

@russellbanks where does the winget manifest live? Currently, I have some local repositories (e.g. for homebrew) which jreleaser overwrites for me. Is winget different? (I was skimming the docs but haven't looked too closely yet)

microsoft/winget-pkgs

Pull requests are just made to winget-pkgs, usually using and created with a tool like Komac, YamlCreate or WinGetCreate.

P.S. I'm a moderator for winget-pkgs so you can ask me any questions

bitspittle commented 1 year ago

@russellbanks Ah thanks, really helpful info! I don't think I'd mind you submitting an initial manifest for me, then, if it doesn't block me from updating or overwriting it later.

I was trying to search randomly through the manifests.... do you know of a manifest out there which uses the new zip type?

FYI latest Kobweb zip is here: https://github.com/varabyte/kobweb/releases/tag/cli-v0.9.11

However, it's not a standalone zip file. The user will need to have Java 11 (or newer) installed on their machine.

Does winget support the idea of specifying "Java 11" as a dependency, at which point it handles installing things automatically? Or would it be up to me to bundle Java 11 into the zip file?

@aalmiray Does this mean when I run jreleaser targeting winget, it won't immediately update anything but will submit a PR on my behalf?

russellbanks commented 1 year ago

Ah, winget doesn't support dependencies currently. It is planned as a "soon" feature. See Dependencies.md on winget-cli.

For the time being, you could make a second zip that has a stripped JRE using JLink bundled with it? I believe that's what Compose does when you run createDistributable (My compose project is on desktop so may just be for that). That's entirely up to you though.

aalmiray commented 1 year ago

@aalmiray Does this mean when I run jreleaser targeting winget, it won't immediately update anything but will submit a PR on my behalf?

Not by default, no. The original idea was to push to an intermediate repository (winget-kobweb for example) and trigger a GH workflow that pull create a branch and push. The PR would still have to be done manually unless the workflow were to use the gh command. This is how https://github.com/jreleaser/jreleaser-macports is handled.

However, I just added an option to specify a branchPush property on taps/repositories (https://github.com/jreleaser/jreleaser/issues/1129) which would give you the ability to directly create a branch on a target repository such as winget-pkgs, bypassing winget-kobweb. The actual PR would still have to be created manually at the moment.

bitspittle commented 1 year ago

Hmmm, I'm thinking I should probably should look into jlink at this point.

Give me a week or so, I have some other things I need to focus on first, but I'll ramp up on jlink / jpackage over the next few days.

Maybe I'll even give in and finally move publishing my artifacts via Jreleaser from a CI instead of my personal desktop, like an actual professional or something :stuck_out_tongue: Once that's migrated, then it wouldn't be too hard to add a workflow that runs on Windows for Winget.

(I just spent a week getting CI artifact support working for two of my other projects. It's a pain, to be honest, and I'm not looking forward to the game of CI trial and error yet again at the moment)

@russellbanks No createDistributable or anything like it in my tasks, sadly. Note that it seems pretty unlikely for Kobweb users to not have Java installed on their machine, since they're doing Kotlin development anyway. Having winget handle the dependencies for me would be the best though, I think.

aalmiray commented 1 year ago

(I just spent a week getting CI artifact support working for two of my other projects. It's a pain, to be honest, and I'm not looking forward to the game of CI trial and error yet again at the moment)

That's why JReleaser has a dry-run mode and offers plenty of options to run as much as possible on a local environment. Alas, when platform specific binaries or tools are required you must test on the given platform.

Skaldebane commented 11 months ago

Hi there! It would be nice to support .rpm packages for Fedora and other RHEL-based Linux distros. Thanks!

Skaldebane commented 11 months ago

Flatpak is also a pretty solid one. It ships with almost every Linux distro by default (barring Ubuntu), and Flathub allows publishing CLI apps (although they won't be shown in the apps store understandably).

bitspittle commented 11 months ago

@Skaldebane thanks! I updated the original comment.

Do you have any experience with either of those package managers? At one point, I checked out snap and it was a wasted day.

Skaldebane commented 11 months ago

@bitspittle I've previously created an .rpm for a specific purpose, but nothing really proper. As for Flatpak, I do have some experience with it (for making a GTK app), but not in the Java ecosystem.

However, I'll try to package Kobweb into one of them when I find some time!

Edit: Both are apparently supported by JReleaser.

aalmiray commented 11 months ago

FWIW JReleaser supports both packagers.

The Native Installer assembler relies on Jpackage to create an .rpm that bundles a Java Runtime along with the app.

The Spec packager OTOH builds an .rpm as expected by Fedora COPR’s build infrastructure.

Lastly, Flatpak may be used to package CLI and/or desktop apps

bitspittle commented 11 months ago

(Ah sorry @aalmiray meant to reply last week!)

Just wanted to say thanks for the last comment. If we can do everything with JReleaser, that's definitely the way to go.

@Skaldebane if you were going to try to experiment with this yourself, the jreleaser config / kobweb binary is actually in a different project. You can find it here. No pressure if you don't have time to work on it; if you ever do, feel free to reach out to me for any help you need.

Ayfri commented 9 months ago

Any update on winget support?

bitspittle commented 9 months ago

Thanks for the reminder @Ayfri . I added it to my near-term todo list to revisit, to see if it will work seamlessly now with latest JReleaser.

I probably have a week or so more of Kobweb work to get through, and then I'll see if I can get winget support working after that. I'll report back here whether it goes well or if there are still obstacles.

bitspittle commented 8 months ago

I probably have a week or so more of Kobweb work to get through, and then I'll see if I can get winget support working after that. I'll report back here whether it goes well or if there are still obstacles.

Just checking in that I've been slammed by a handful of surprises after posting this comment, so my week estimate has apparently ballooned into a month. However, I haven't forgotten this issue, and I am hoping to take a look into it later this week or next.

Ayfri commented 8 months ago

Haha I see, no problem, unforeseen events happen to anyone !

bitspittle commented 7 months ago

Just an update to my winget folks, I do not think the attempt to support winget could be going more poorly.

I've got a bunch of other tasks on my plate, so I'm going to abandon this for now. Apologies at it seems like this is the most requested package manager at the moment.

I will enumerate my concerns below. People with more knowledge of the matter can certainly point out flaws in my thinking or outright mistakes.

My recommendation at this point for Windows folks is to just use scoop or sdkman.

If winget would consider supporting external manifests added manually via remote repositories, and if it wouldn't incorrectly report my code has malware, AND if I could figure out why I was getting those weird "This software is not compatible with 64-bit windows", then maybe I could be convinced to revisit winget support in the near future.

aalmiray commented 7 months ago

@bitspittle I've raised https://github.com/jreleaser/jreleaser/issues/1596 and https://github.com/jreleaser/jreleaser/issues/1597 for the first 2 items.

Using branchPush allows you to create a branch other than the current one when pushing to a given repository. JReleaser lets you by default to push to a winget package repository of your own. It would be your responsibility to create a PR to the shared package repository on your own.

By using the branchPush you may be able (if access rights have been granted) to create such branch directly on the shared package repository. Take note that JReleaser does not yet provides the means to create PRs automatically. Even if a branch were to be created in this way you'd still have to manually create the PR.

The only advantage of using branchPush in this way is to skip your own package repository.

aalmiray commented 7 months ago

One more thing, the report as malware is likely caused by an unsigned package. https://github.com/jreleaser/jreleaser/issues/689 keeps track of this issue

bitspittle commented 7 months ago

Thank you, as always @aalmiray , for the insanely fast response and turnaround time.

If the malware error comes from the lack of signing, it's a shame that winget isn't taking the 256 sha into account. I mean, I'm essentially confirming that my artifacts are hosted in my own repo. I know that just because you're hosting open source software on github, that doesn't mean it can't have malware, but then again, I could always sign my own software which has malware in it, so it's not like it provides that much of an extra benefit as far as I can tell.

That said, I have set up signing for maven in a bunch of places, so if it was clear to me exactly what file winget wanted me to sign and where that key should be found, I could look into adding it, assuming it's some standard signing technology and not something that is windows only. (I have gpg set up in other projects of mine)

aalmiray commented 7 months ago

😅 Thanks. It was fun adding what was missing for winget.

Regarding signing, if this is indeed the case then I'm afraid you must use Windows' signtool which is setup way differently to PGP (what Maven Central requires at the moment).

Treetrain1 commented 6 months ago

Would you consider supporting an AUR package?

There is a package on the AUR, but it's very out of date. The latest supported version is 0.9.12.

aalmiray commented 6 months ago

@bitspittle you may recall we have pending support for AUR in JReleaser https://github.com/jreleaser/jreleaser/issues/693

bitspittle commented 6 months ago

@aksh1618 are you still around? The AUR work you did for Kobweb so many years ago broke when I had to move the kobweb CLI out into a different repository. Would it be easy to fix up the script you wrote to use this new repository location?

aksh1618 commented 6 months ago

Hey @bitspittle! Ah I see, I think the action should work in this repository as well without too many changes, let me double check and get it back up this weekend. I'll reach out to you on discord if I need anything w.r.t. permissions etc.