Packaging and uploading MSI installer for Windows

[mislav]

We are currently using goreleaser to package and distribute a CLI app. Thank you for this great tool!

For Windows, we need to package, sign, and upload an MSI to a release. Since this is highly platform-specific, I'm not suggesting that goreleaser implements any part of that workflow (seeing how creating macOS installer packages was closed as wontfix), but in a similar manner to how several small features were added to facilitate signing and notarizing macOS apps, I'm wondering whether you would consider adding similar tweaks to allow us to configure goreleaser to create custom archives.

On top of existing archive formats (tar.gz, zip), it would be useful if there was a custom format which would invoke the given command and add the file produced to release uploads. We could then use that to invoke go-msi to generate the "archive".

Right now, no combination of build hooks or signing configuration seems to have the ability to wrap a built binary in a new package and have that file uploaded to a release.

The signing feature does seem to be able to create a new arbitrary file via signature: "${artifact}_sig" that is subsequently uploaded, but if we use that in combination with binary archive format and name the signature file "${artifact}.msi", we would end up with a file named foo.exe.msi on Windows (with seemingly no ability to strip the .exe bit), and the original foo.exe would still be uploaded, which we do not want since the MSI is preferred.

---

For Windows we are forced to side-step Goreleaser completely for now and construct our own pipeline to build the MSI and attach it to an existing release. Since running a subset of builds is also not supported, we have to duplicate our build configuration (including a tricky ldflags setup) and implement uploading of the new package to a release. We have done all this and it works, but if goreleaser was a tad more configurable, we could have saved considerable effort.

Thank you for reading! 🙇

[caarlos0]

If go-msi works on linux and mac as well, maybe we can add it natively (even better if it has a Go API - I haven't looked into it yet though).

If we could make it an archive format, I think it would solve all problems here.

The foo.exe.msi on the sign part seems like a bug to me... will have a look.

On top of existing archive formats (tar.gz, zip), it would be useful if there was a custom format which would invoke the given command and add the file produced to release uploads. We could then use that to invoke go-msi to generate the "archive".

this also seems like a good idea 🤔

I'll think about this for a couple of days, feel free to ping me if I don't get back to it soon :)

[mislav]

If go-msi works on linux and mac as well

I don't see a reason why go-msi itself wouldn't work on non-Windows platforms, but the tool itself is basically a wrapper around invoking individual WiX Toolset binaries, and those are only available on Windows (to my knowledge). So, I think that with this approach, it's only feasible to create MSI by running the job on Windows.

If this worked on all platforms, we would be more than happy to contribute a feature to Goreleaser that packages and signs MSIs, as that would make life easier for a lot of people distributing software on Windows.

The foo.exe.msi on the sign part seems like a bug to me... will have a look.

I created a repro here https://github.com/mislav/goreleaser-test

In addition to generating the .exe.msi filename, one thing is also odd as well:

   • BUILDING BINARIES
      • building                  binary=dist/foo_linux_amd64/foo
      • building                  binary=dist/windows-build_windows_amd64/foo.exe
   • ARCHIVES
      • skip archiving            binary=foo.exe
      • creating                  archive=dist/foo_0.0.1_linux_amd64.tar.gz
...
   • SIGNING ARTIFACTS
      • signing                   cmd=[./signtool dist/windows-build_windows_amd64/foo.exe.msi dist/windows-build_windows_amd64/foo.exe]
      • signing                   cmd=[./signtool dist/foo_0.0.1_checksums.txt.msi dist/foo_0.0.1_checksums.txt]

I've expected signing to only happen for the Windows build since I've used the new ids: feature, but the tool for signing was invoked for both foo.exe and foo_0.0.1_checksums.txt. If this was a real tool that signs Windows binaries, it would have choked on the checksums.txt file. Is this by design?

Thank you for taking this into consideration! 🌟

GitHub

mislav/goreleaser-test

Contribute to mislav/goreleaser-test development by creating an account on GitHub.

[caarlos0]

If this was a real tool that signs Windows binaries, it would have choked on the checksums.txt file. Is this by design?

artifacts: all includes the checksum as well, we'll probably need to add another option to sign only the binaries... 🤔

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[caarlos0]

not stale

[abemedia]

Any update on this? On OSX and Linux there's a package called msitools which allows creating an msi https://wiki.gnome.org/msitools/HowTo/CreateMSI. It uses the same XML format as WiX. Potentially an XML could be generated, then sent to wixl (the msitools compiler) on Linux/OSX or WiX toolset's candle & light if run on windows. I like the idea of it being an archive format @caarlos0 , however that could be tricky as it requires a bunch of options that other archives don't (GUIDs for updates, start menu options etc.)

[caarlos0]

no updates yet.

---

about msitools: if it works on all platforms, we may add it as its own pipe (like we have brew today) and if someone wants to work on it and help maintain it later I might consider accepting it.

[abemedia]

Awesome. Will see if I can make the time to have a bash at this over the next few days. Is there any existing pipe you'd recommend which could serve as a basis for adapting?

[mislav]

On OSX and Linux there's a package called msitools which allows creating an msi https://wiki.gnome.org/msitools/HowTo/CreateMSI. It uses the same XML format as WiX.

Thanks for sharing! Can the resulting MSI then also be signed on macOS/Linux? For us—and I'm guessing for anyone else who wants to distribute their MSIs to Windows users—it's little useful that we're able to generate a MSI on a platform where we are not able to also sign them. Just something to keep in mind when developing the goreleaser pipe.

[abemedia]

@mislav this can be done using Mono: https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Build_Instructions/Signing_an_executable_with_Authenticode

[42wim]

Maybe NSIS is an option instead of msi? Can be "easily" build from linux. https://www.conjur.org/blog/building-a-windows-installer-from-a-linux-ci-pipeline/ https://nsis.sourceforge.io/Main_Page

[joe-getcouragenow]

There is golang packaging code for Windows, Mac and Linux here: https://github.com/go-flutter-desktop/hover/tree/master/cmd/packaging

I use it for flutter golang. But it can just as easily be used to package any golang project. I think it would work well with a few changes. Not sure if a fork makes sense or not.

It also does bundle process.

hover init-packaging -h
Create configuration files for a packaging format

Usage:
  hover init-packaging [command]

Available Commands:
  darwin-bundle  Create configuration files for OSX bundle packaging
  darwin-dmg     Create configuration files for OSX dmg packaging
  darwin-pkg     Create configuration files for OSX pkg installer packaging
  linux-appimage Create configuration files for AppImage packaging
  linux-deb      Create configuration files for deb packaging
  linux-pkg      Create configuration files for pacman pkg packaging
  linux-rpm      Create configuration files for rpm packaging
  linux-snap     Create configuration files for snap packaging
  windows-msi    Create configuration files for msi packaging

hover build -h
Build a desktop release

Usage:
  hover build [command]

Available Commands:
  darwin         Build a desktop release for darwin
  darwin-bundle  Build a desktop release for darwin and package it for OSX bundle
  darwin-dmg     Build a desktop release for darwin and package it for OSX dmg
  darwin-pkg     Build a desktop release for darwin and package it for OSX pkg installer
  linux          Build a desktop release for linux
  linux-appimage Build a desktop release for linux and package it for AppImage
  linux-deb      Build a desktop release for linux and package it for deb
  linux-pkg      Build a desktop release for linux and package it for pacman pkg
  linux-rpm      Build a desktop release for linux and package it for rpm
  linux-snap     Build a desktop release for linux and package it for snap
  windows        Build a desktop release for windows
  windows-msi    Build a desktop release for windows and package it for msi

It can also do the same with docker

It also able to add extra dependencies into the packaging. Like .ddl, .so, framework etc.

It also does the icon generation.

[joe-getcouragenow]

correction to above: I should have said that its written in golang and can be used to package any code that is written in any language.

so its agnostic..

Has anyone moved on this yet or are there any further thoughts on this approach ?

I think that code is a good basis and its been reasonably battle tested over the last year. Will have a few kinks - for example there is so bug with darwin pkg installs.

i use it with gon for signing apple stuff. For windows, it has wix support, but i have not tried signing.

[abemedia]

That package shells out to wix/msitools.

[joe-getcouragenow]

@abeMedia Yep. YOu have to manually install those tools your self.

See why they did not include them here: https://github.com/go-flutter-desktop/go-flutter/issues/522

A PR to hover could easily add the ability to download the wix tools for the developer as part of bootstrapping.. Just has to download from github release here, and then kick off the install for the developer. OR just wrap hover and write some golang to install it before the hover packager is called.

The code that then does the packaging for windows is here: https://github.com/go-flutter-desktop/hover/blob/master/cmd/packaging/windows-msi.go#L26

Here it calls Candle: https://github.com/go-flutter-desktop/hover/blob/master/cmd/packaging/windows-msi.go#L62

[abemedia]

Don't see what the benefit of using that package is then...

[joe-getcouragenow]

@abeMedia why though. It does a ton of work and does it really well. It can package anything. Does not have to be golang or flutter

[caarlos0]

Any MSI experts here?

https://github.com/goreleaser/nfpm/discussions/345

[caarlos0]

This would allow:

• winget support #2290
• chocolatey support #1963

[Inveracity]

Looking at a package like this grep one

• https://community.chocolatey.org/packages/grep

it's possible to download the .nupkg file and rename it to .zip

after unzipping it I had a look at the contents:

grep
├── [Content_Types].xml
├── _rels
├── grep.nuspec
├── package
│   └── services
│       └── metadata
│           └── core-properties
│               └── 32782d237eb94319a187b187abf4a906.psmdcp
└── tools
    ├── LICENSE.txt
    ├── VERIFICATION.txt
    ├── chocolateyinstall.ps1
    ├── chocolateyuninstall.ps1
    └── install
        └── grep-windows-3.7
            ├── README.md
            └── grep-3.7-x64.exe

Looking through the files a lot of this could become templated files where values like name, version, author, description can be injected. So when goreleaser wants to make a chocolatey package it could potentially just generate all the required files, make a zip archive and rename that zip archive to .nupkg and upload it?

Pure speculation, haven't tried it.

[caarlos0]

nsis seems like a good alternative, althoug my understanding is that it does not generate msi files, but installables nonetheless.

seems like we could have a homebrew kind of templated script with the very basics (accept license, copy files, and maybe uninstall), or allow a custom script and just replace some variables there (which I believe would probably be easier, more like the docker pipe works).

once that's working, we could add winget/chocolatey integration too.

any thoughts/opinions/concerns?

[rfay]

nsis is pretty ancient, but also pretty simple. You can build and push with a docker image, etc. And you don't need Windows for any of it.

[caarlos0]

using nsis would, of course, add nsis as a dependency to anyone using the nsis pipe (or installer pipe)

and in that sense, maybe we could have a more generic "windows_installer" pipe, that can use either wix or nsis (or anything else?) after applying templates to a set of config files (i.e. set the version, dates, etc).

[caarlos0]

to be fair, I would still prefer to do all in "native go", but if its no possible, its no possible... not going to fight too much about it.

People really need installers for windows, it seems 😄

[rfay]

It's really hard on Windows to "just put the binary in /usr/local/bin". Really hard :(

[twpayne]

Hopefully the situation is improving. https://github.com/microsoft/winget-cli/issues/182 just got closed, we're waiting for documentation on how to use it now.

[caarlos0]

this might not be needed anymore, check https://github.com/goreleaser/goreleaser/pull/4081

[briantist]

winget is not a complete eleventh replacement for MSI

[caarlos0]

winget is not a complete eleventh replacement for MSI

yep, I'm aware, but afaik part of the reason people wanted MSI support was to publish it to winget... if so, that's not needed anymore :)

[mislav]

As a person who originally opened this issue and has since then maintained multiple iterations of GitHub CLI's approach for building an MSI for each of our releases, I would also recommend that most CLI tools avoid making an MSI and instead have their users install the tool by having something like winget extract a zip archive.

Creating an installer requires a large amount of boilerplate; this is true both for NSIS (which I understand are limited in functionality) and for regular MSI. We use MSBuild.exe plus WiX Toolset for creating installers, which is sort of the industry standard by now, but the tooling around all this is atrocious and the whole process requires setting up arcane-looking XML files. I will forever be grateful to @heaths for helping set this up for the GitHub CLI project, but I wouldn't inherit the same approach for other projects, nor do I believe that it is a good aim for the Goreleaser project to ever support creating these installers, at least not until someone creates a Go library for generating these installers on any platform and without boilerplate.

[twpayne]

For info, chezmoi is built with goreleaser and made available as a Winget package via winget-releaser. With this path, an MSI is not needed. See https://github.com/twpayne/chezmoi/pull/2761 if you want to copy the workflow for your own project.

[twpayne]

Ah, and now I see that goreleaser has its own support for Winget. Apologies for the noise.

[caarlos0]

this was released in goreleaser v1.24.0-pro https://goreleaser.com/customization/msi/