search

twpayne / chezmoi

Manage your dotfiles across multiple diverse machines, securely.
https://www.chezmoi.io/
MIT License
13.36k stars 493 forks source link

Add winget package #812

Closed twpayne closed 1 year ago

twpayne commented 4 years ago

Describe the solution you'd like

chezmoi should be installable with winget.

Describe alternatives you've considered

Installing chezmoi with chocolatey, scoop, or manually.

Additional context

winget requires an installer. https://github.com/goreleaser/goreleaser/issues/1295 tracks adding support for this to goreleaser.

bradenhilton commented 2 years ago

Not sure if you've seen this, but you might not need an installer soon™.

twpayne commented 2 years ago

As an update on this, this is not something that is chezmoi-specific. We're waiting for either winget to support installing utilities from archives (e.g. https://github.com/microsoft/winget-cli/issues/182), or GoReleaser to generate MSIs (https://github.com/goreleaser/goreleaser/issues/1295). If you want to help, please contribute to either of these issues.

bradenhilton commented 2 years ago

Small update: Winget merged the spec in https://github.com/microsoft/winget-cli/pull/2012. I think we need to wait for the next release now.

twpayne commented 2 years ago

Awesome, thank you for tracking this.

twpayne commented 2 years ago

Related to this, 5a143268e7f623b8d93dd47d0055d5b6d708e5fe added generating raw (unpackaged) binaries to chezmoi's releases, i.e. you can down directly download a .exe from chezmoi's releases page.

jthoward64 commented 2 years ago

Small update: Winget merged the spec in microsoft/winget-cli#2012. I think we need to wait for the next release now.

In Preview: https://github.com/microsoft/winget-cli/releases/tag/v1.3.1251-preview

latipun7 commented 2 years ago

Small update: Winget merged the spec in microsoft/winget-cli#2012. I think we need to wait for the next release now.

It's already in stable release for quite some times: https://github.com/microsoft/winget-cli/releases/tag/v1.3.2091 Example github workflows: https://github.com/jandedobbeleer/oh-my-posh/blob/main/.github/workflows/winget.yml

twpayne commented 2 years ago

Thanks for the link. I did a bit of digging into jandedobbeleer/oh-my-posh and I think it's generating winget packages from an installer. Specifically:

So, I don't think jabdedobbeleer/oh-my-posh is using winget with ZIP files or raw .exes.

If anyone has time to spend on this together (my understanding of Windows and winget is very limited, but I know chezmoi's release process quite well) then I'd be very happy to work on this together.

bradenhilton commented 2 years ago

@twpayne correct, the winget package for jandedobbeleer/oh-my-posh uses Inno: https://github.com/microsoft/winget-pkgs/blob/218451e5f3feae0191d6b1d7417ac21689735219/manifests/j/JanDeDobbeleer/OhMyPosh/8.9.2/JanDeDobbeleer.OhMyPosh.installer.yaml#L9

Based on https://github.com/microsoft/winget-pkgs#authoring-a-manifest:

There are 3 primary options for creating the manifest:

wingetcreate:

YamlCreate.ps1 (documentation):

Substituting values in a manually created manifest:

twpayne commented 2 years ago
  • I tried to create a manifest with it and got a vague error, not sure why yet:

Is this error from parsing the URL or the contents of the .exe file? My admittedly very poor understanding of the error message is that wingetcreate expected to download an MSI installer (which it could then unpack for files) but instead found a .exe file that looked nothing like an MSI installer.

bradenhilton commented 2 years ago

@twpayne I opened an issue to see if we can get to the bottom of it.

What I can say is it tries to detect some information from the binary which is not available (perhaps a limitation of goreleaser? I tried to see if it's possible to set this information in built binaries and didn't find anything), but my understanding is that it should ask for user input for anything it can't determine automatically instead of just failing outright.

chezmoi binary ![image](https://user-images.githubusercontent.com/23157013/188728605-740797a5-e091-429f-91a7-fc4bbb47e92d.png)

For contrast, here's a Discord installer:

Discord installer binary ![image](https://user-images.githubusercontent.com/23157013/188728951-1195d0ba-348f-47c7-9180-aa6464529650.png)

Note that none of this means winget support is impossible, we could always fall back to substituting values in manifest templates.

twpayne commented 2 years ago

Thank you for the investigation! The chezmoi binary is currently just the chezmoi program itself, not an installer like DiscordSetup.exe.

bradenhilton commented 2 years ago

I am aware of this, I just picked it because it was right next to the chezmoi binary in my downloads folder. The intention was just to provide an example of what it looks for by default. Those fields are not only populated on installers.

See Nvidia ICAT for another example, which I believe is a standalone binary:

Nvidia ICAT binary ![image](https://user-images.githubusercontent.com/23157013/188736084-77199326-2d01-428b-a632-39316b1d4a68.png)

Looks like VERSIONINFO is responsible for it.

Following https://stackoverflow.com/a/66702321/11000543, I am able to force-populate the fields:

chezmoi binary via ResourceHacker ![image](https://user-images.githubusercontent.com/23157013/188738352-0288ff08-7f70-49cb-9277-d4742e331bde.png)

I wouldn't at all be surprised if populating the fields is impossible with goreleaser.

twpayne commented 2 years ago

Sorry for my assumptions here - I was wrong.

I suspect that the population of VERSIONINFO should be done by the go toolchain and that goreleaser has no control over this (goreleaser invokes go build for all OSes and architectures, but does not otherwise manipulate the resulting binaries, if my understanding is correct). I don't know if populating VERSIONINFO is currently possible. Maybe it would be possible with something like Go 1.18's debug/buildinfo.

bradenhilton commented 2 years ago

Perhaps GoVersionInfo is sufficient for this. It even supports embedding an icon.

rclone is using GoVersionInfo.

dpprdan commented 2 years ago

Another issue might be that chezmoi's Windows executables are not signed (AFAICT). Maybe the gh release workflow helps? They use goreleaser as well and sign executables with https://github.com/mtrojnar/osslsigncode AFAICT.

or GoReleaser to generate MSIs (goreleaser/goreleaser#1295)

The workflow ☝🏻 uses a separate job to build and sign MSIs.

bradenhilton commented 2 years ago

@dpprdan https://github.com/twpayne/chezmoi/pull/2394

twpayne commented 2 years ago

The release signing in #2394 uses cosign, whereas Windows has its own signing system which we'd need to use.

dpprdan commented 2 years ago

Another issue might be that chezmoi's Windows executables are not signed (AFAICT).

Here is a post that describes the process in detail: Automatic Code-signing on Windows using GitHub Actions

twpayne commented 2 years ago

Thanks for the link. According to that I need to pay $70/year and maintain a self-hosted Windows GitHub Actions runner. This is more than I am prepared to do.

dpprdan commented 2 years ago

Understandable.

Sidenote: I am not entirely sure whether a self-hosted Windows GitHub Actions is strictly necessary, though probably more secure? At least the gh cli release workflow doesn't seem to use one, AFAICT. But I'm generally out of my depth here.

bradenhilton commented 2 years ago

Are we absolutely sure signing is necessary for portable apps?

I think we should just manually create a manifest and see if it gets accepted so we can know once and for all.

jonaskuske commented 2 years ago

Other than that, distribution through the Microsoft Store probably is an option too, then you don't need a certificate as Microsoft does the signing. And msstore is one of the sources of winget, so you can install through winget CLI too. Don't know if there's a good MSIX packager for Go apps though.

dpprdan commented 2 years ago

Don't know if there's a good MSIX packager for Go apps though.

AFAIU that can be done in a separate job after building the exe with goreleaser

bradenhilton commented 1 year ago

Just seen vedantmgoyal2009/winget-releaser in my explore section, looks like it's an action wrapper around YamlCreate.ps1.

sitiom commented 1 year ago

Winget 1.4 has just released, adding support for zip installation. I already created a PR for chezmoi:

Once this is merged, the Winget Releaser workflow as mentioned above should be used to set up automatic package updates.

twpayne commented 1 year ago

Awesome, thank you very much :)

bradenhilton commented 1 year ago

Zip installation wasn't the block for this issue, it was blocked by a lack of a sufficient manifest generation and PR automation tool. We could have manually written an initial manifest months ago when portable exe file support was added to the spec, the problem was keeping it updated. I'm unsure why zip was chosen over portable.

I'd just picked this back up over the last few days because my issue in wingetcreate was closed with an incomplete fix, and it hasn't yet been reopened or received any kind of response.

I thought it might have something to do with the exe missing a manifest, so I added a basic one and it still didn't work.

wingetcreate doesn't parse the exe (or zip) correctly, and YamlCreate.ps1 requires manually cloning your personal winget-pkgs fork to create the PR for every release (which is certainly possible, but quite clunky).

@sitiom the package identifier should ideally be chezmoi.chezmoi IMO, there are plans to move this repo to the chezmoi organization eventually.

sitiom commented 1 year ago

it was blocked by a lack of a sufficient manifest generation and PR automation tool

WingetCreate and YamlCreate.ps1 have already existed a long time ago.

I'm unsure why zip was chosen over portable.

The only unzipped executable in the releases is amd64 (https://github.com/twpayne/chezmoi/releases/download/v2.29.4/chezmoi-windows-amd64.exe). The zip files cover all architectures.

I'd just picked this back up over the last few days because my issue in wingetcreate was closed with an incomplete fix, and it hasn't yet been reopened or received any kind of response.

YamlCreate.ps1 is better based on my experience 😅. Please have a look at Komac as well, which is currently under development.

YamlCreate.ps1 requires manually cloning your personal winget-pkgs fork to create the PR for every release (which is certainly possible, but quite clunky).

Winget Releaser automates this exact use case. It's convenient. See examples: microsoft/winget-pkgs/pulls (Pull request has been automatically created using 🛫 WinGet Releaser).

@sitiom the package identifier should ideally be chezmoi.chezmoi IMO, there are plans to move this repo to the chezmoi organization eventually.

Sure, when is this? Can you point me to the specific discussion?

bradenhilton commented 1 year ago

it was blocked by a lack of a sufficient manifest generation and PR automation tool

WingetCreate and YamlCreate.ps1 have already existed a long time ago.

I am aware of this, I explained further in my comment why they are insufficient (wingetcreate) and clunky (YamlCreate.ps1).

I'm unsure why zip was chosen over portable.

The only unzipped executable in the releases is amd64 (https://github.com/twpayne/chezmoi/releases/download/v2.29.4/chezmoi-windows-amd64.exe). The zip files cover all architectures.

Good point.

YamlCreate.ps1 requires manually cloning your personal winget-pkgs fork to create the PR for every release (which is certainly possible, but quite clunky).

Winget Releaser automates this exact use case. It's convenient. See examples: microsoft/winget-pkgs/pulls (Pull request has been automatically created using 🛫 WinGet Releaser).

I am aware of this also, but was unsure of where the winget-pkgs fork should be set up to be used within the workflow.

@sitiom the package identifier should ideally be chezmoi.chezmoi IMO, there are plans to move this repo to the chezmoi organization eventually.

Sure, when is this? Can you point me to the specific discussion?

An email exchange between myself and @twpayne on 2022-06-09:

At some point we should move github.com/twpayne/chezmoi to github.com/chezmoi/chezmoi, but not quite yet :)

That said, it's only an ideal. We can move the manifest when needed, correct?

sitiom commented 1 year ago

I am aware of this also, but was unsure of where the winget-pkgs fork should be set up to be used within the workflow.

From the readme:

Fork the winget-pkgs repository under the same account/organization as your repository on which you want to use this action.

That said, it's only an ideal. We can move the manifest when needed, correct?

Yes, that would be easy.

twpayne commented 1 year ago

Forgive any errors in the following, as my understanding of Windows and Winget is very shaky.

If it helps, I can modify the release process to also include Windows arm, arm64, and i386 .exe files assets, not just .zips. Is this needed/helpful?

If I understand correctly, I'll need to folk microsoft/winget-pkgs to twpayne/winget-pkgs. Is this correct?

Will using Winget Releaser mean that a $70/year signing key and dedicated GitHub Actions runner are still needed?

sitiom commented 1 year ago

If it helps, I can modify the release process to also include Windows arm, arm64, and i386 .exe files assets, not just .zips. Is this needed/helpful?

Now that zips are supported, not really much but go ahead. 1.4 validation in the pipelines hasn't been turned on, so you would either wait for a few days or push the .exe file assets now so the PR can be merged faster.

If I understand correctly, I'll need to folk microsoft/winget-pkgs to twpayne/winget-pkgs. Is this correct?

Yes. I can help add the winget releaser workflow in a PR, but only after the package has already been added to Winget.

Will using Winget Releaser mean that a $70/year signing key and dedicated GitHub Actions runner are still needed?

You would need to use GitHub actions for Winget Releaser (it is an action). You don't need a signing key at all; a lot of Winget packages have unsigned binaries.

twpayne commented 1 year ago

Thanks for the info.

Let's wait for the 1.4 validation to be enabled. Once the https://github.com/microsoft/winget-pkgs/pull/94759 is merged I'll fork winget-pkgs and we can work on the automation.

twpayne commented 1 year ago

Huge thanks to @sitiom and @bradenhilton for their work in getting chezmoi into winget: https://github.com/microsoft/winget-pkgs/pull/94759 https://github.com/twpayne/chezmoi/pull/2735

Let's keep this issue open until we have automatic updates of winget packages on each chezmoi release.

sitiom commented 1 year ago