electron-userland / electron-builder

A complete solution to package and build a ready for distribution Electron app with “auto update” support out of the box
https://www.electron.build
MIT License
13.69k stars 1.74k forks source link

Unable to sign NSIS #7973

Open idoodler opened 10 months ago

idoodler commented 10 months ago

I am tryingg to build our Windows app in Docker, however I am always getting the following signing error:

⨯ Exit code: 255. Command failed: /root/.cache/electron-builder/winCodeSign/winCodeSign-2.6.0/linux/osslsigncode -in **REDACTED**E.exe -out **REDACTED**-signed-sha256.exe -t http://timestamp.digicert.com -pkcs12 ./build/**REDACTED**.p12 -h sha256 -n **REDACTED** -i **REDACTED** -nest -pass **REDACTED** (sha256 hash) to extract existing signature in -nest mode

We are using gitlab, here is the part ofthe .gitlab.yml

Windows:
    stage: build
    tags:
        - docker
    image: electronuserland/builder:wine
    script:
        - yarn
        - yarn run gsf:download --secretId 6 -o apps/raptor-electron # downloads the electron-builder.env
        - yarn workspace @**REDACTED**/raptor-electron run compile -w # Here it stops with the exception
        - mv 'apps/raptor-electron/dist/**REDACTED**.exe' './'
        - mv 'apps/raptor-electron/dist/**REDACTED**.exe.blockmap' './'
        - yarn workspace @**REDACTED**/raptor-electron run compile:appx
        - mv 'apps/raptor-electron/dist/**REDACTED**.appxbundle' './'
    artifacts:
        paths:
            - **REDACTED**.exe
            - **REDACTED**.exe.blockmap
            - **REDACTED**.appxbundle
    rules:
        - if: $NIGHTLY =~ /raptor/
          when: on_success
        - changes:
              - apps/raptor/**/*
              - apps/raptor-electron/**/*
          when: on_success

I first thought that the password of the Certificate file is wrong, but it is verified and correct. We are using the electron-builder.env file in the projects root directory which is downloaded by the build script. As a key for the Certificate File I am using WIN_CSC_KEY_PASSWORD

The job is executed on a Debian Machine

mmaietta commented 10 months ago

Are you able to repro this on a local dev environment? Wondering if it's CI-specific

idoodler commented 10 months ago

@mmaietta Indeed, I just set it up locally and it also won't work bare on macOS. I compared it to another project and the second SHA256 signing is not working. The passphrase hash is equal to the one in the project in question, so I am 100% sure the password is correct, also the SHA1 based signature is working.

Here the log with DEBUG=*


  • exited          command=app-builder_arm64 code=0 pid=4870
  • signing         file=dist/win-unpacked/REDACTED.exe certificateFile=./build/REDACTED.p12
  • spawning        command=/Users/david/WebstormProjects/monorepo/node_modules/app-builder-bin/mac/app-builder_arm64 download-artifact --name winCodeSign
  • found existing  path=/Users/david/Library/Caches/electron-builder/winCodeSign/winCodeSign-2.6.0
  • exited          command=app-builder_arm64 code=0 pid=4894 out=/Users/david/Library/Caches/electron-builder/winCodeSign/winCodeSign-2.6.0
  • executing       file=/Users/david/Library/Caches/electron-builder/winCodeSign/winCodeSign-2.6.0/darwin/10.12/osslsigncode args=-in /Users/david/WebstormProjects/monorepo/apps/raptor-electron/dist/win-unpacked/REDACTED.exe -out /Users/david/WebstormProjects/monorepo/apps/raptor-electron/dist/win-unpacked/REDACTED-signed-sha1.exe -t http://timestamp.digicert.com -pkcs12 ./build/REDACTED.p12 -h sha1 -n REDACTED -i https://REDACTED.com -pass REDACTED (sha256 hash) env={
  "DYLD_LIBRARY_PATH": "/Users/david/Library/Caches/electron-builder/winCodeSign/winCodeSign-2.6.0/darwin/10.12/lib"
}
  • executed        file=/Users/david/Library/Caches/electron-builder/winCodeSign/winCodeSign-2.6.0/darwin/10.12/osslsigncode stdout=Succeeded

  • executing       file=/Users/david/Library/Caches/electron-builder/winCodeSign/winCodeSign-2.6.0/darwin/10.12/osslsigncode args=-in /Users/david/WebstormProjects/monorepo/apps/raptor-electron/dist/win-unpacked/REDACTED.exe -out /Users/david/WebstormProjects/monorepo/apps/raptor-electron/dist/win-unpacked/REDACTED-signed-sha256.exe -t http://timestamp.digicert.com -pkcs12 ./build/REDACTED.p12 -h sha256 -n REDACTED -i https://REDACTED.com -nest -pass REDACTED (sha256 hash) env={
  "DYLD_LIBRARY_PATH": "/Users/david/Library/Caches/electron-builder/winCodeSign/winCodeSign-2.6.0/darwin/10.12/lib"
}
  ⨯ Exit code: 255. Command failed: /Users/david/Library/Caches/electron-builder/winCodeSign/winCodeSign-2.6.0/darwin/10.12/osslsigncode -in /Users/david/WebstormProjects/monorepo/apps/raptor-electron/dist/win-unpacked/REDACTED.exe -out /Users/david/WebstormProjects/monorepo/apps/raptor-electron/dist/win-unpacked/REDACTED-signed-sha256.exe -t http://timestamp.digicert.com -pkcs12 ./build/REDACTED.p12 -h sha256 -n REDACTED -i https://REDACTED.com -nest -pass REDACTED (sha256 hash) to extract existing signature in -nest mode
Failed

Unable to extract existing signature in -nest mode
Failed
  failedTask=build stackTrace=Error: Exit code: 255. Command failed: /Users/david/Library/Caches/electron-builder/winCodeSign/winCodeSign-2.6.0/darwin/10.12/osslsigncode -in /Users/david/WebstormProjects/monorepo/apps/raptor-electron/dist/win-unpacked/REDACTED.exe -out /Users/david/WebstormProjects/monorepo/apps/raptor-electron/dist/win-unpacked/REDACTED-signed-sha256.exe -t http://timestamp.digicert.com -pkcs12 ./build/REDACTED.p12 -h sha256 -n REDACTED -i https://REDACTED.com -nest -pass REDACTED (sha256 hash) to extract existing signature in -nest mode
Failed

Unable to extract existing signature in -nest mode
mmaietta commented 9 months ago

Hmmm, I haven't seen this issue before and the signing code hasn't changed in a very long time that could impact this (I'm not familiar with any changes to this area of code since I first joined as a maintainer for this project years ago during COVID). I'd be more worried if this was a widespread issue, especially since there's unit tests for windows code signing, albeit it uses a self-signed cert. I'll need to check which GHA runners are being used for that test once I'm no longer swamped with work

Nantris commented 9 months ago

Thanks to both of you for your contributions here. I wonder if there's any update on this from either end?

I'm looking at potentially buying some hardware and setting up a dedicated machine for local GitHub Actions including building/signing for Windows (including .appx) inside of the Docker and want to make sure that's actually possible first.

idoodler commented 9 months ago

@Slapbox I didn't further investigate this as i currently have to prioritize other things at work.

Tho I'd definitely like to help investigate this if someone needs information.

mmaietta commented 9 months ago

Curious, instead of the env vars (wondering if there's some parsing issue there), can you try providing your env config directly into the Windows Options build configuration? https://github.com/electron-userland/electron-builder/blob/04f5784730112255a21c448bc87957ee66c1d53f/packages/app-builder-lib/src/options/winOptions.ts#L35-L44

I checked the CI and it only tests Windows code signing on a Windows GHA runner, so I might need to extend that to also test on Linux for this (probably standard) use case.

mmaietta commented 9 months ago

Bingo, I think I can investigate further from here. But maybe a minimum reproducible repo would also help as the unit tests are super complex and a vanilla repo would probably really help here. Would you be willing to put one together?

https://github.com/electron-userland/electron-builder/actions/runs/7729511593/job/21072858970?pr=8023

  ● sign nested asar unpacked executables
    Error: expect(received).toContain(expected) // indexOf
    Expected substring: "This file format cannot be signed because it is not recognized."
    Received string:    "Exit code: 255. Command failed: /root/.cache/electron-builder/winCodeSign/winCodeSign-2.6.0/linux/osslsigncode -in /tmp/et-db411cd21f3bbd882f3a5a30c58db6ff/t-aczwLS/test-project-0/dist/win-unpacked/Test App ßW.exe -out /tmp/et-db411cd21f3bbd882f3a5a30c58db6ff/t-aczwLS/test-project-0/dist/win-unpacked/Test App ßW-signed-sha256.exe -t http://timestamp.digicert.com -pkcs12 /tmp/et-db411cd21f3bbd882f3a5a30c58db6ff/t-aczwLS/1.p12 -h sha256 -n Test App ßW -i http://foo.example.com -nest
    Unable to extract existing signature in -nest mode

    Failed

    Unable to extract existing signature in -nest mode
mmaietta commented 9 months ago

Can you give this sign.js script for win.sign config as a test of my current assumption?

const { doSign } = require('app-builder-lib/out/codeSign/windowsCodeSign')

/**
 * @type {import("electron-builder").CustomWindowsSign} sign
 */
module.exports = async function sign(config, packager) {
  config.isNest = false
  await doSign(config, packager)
}

I think what isNest is doing is that it's appending to file signatures, which don't seem to exist within a docker image? Disabling isNext I think will bypass this and only update/create a signature

Nantris commented 9 months ago

Thanks for investigating this @mmaietta! Is there any potential harm in using that script when building natively on Windows? We're at sort of a transitional stage between native builds and Docker builds.

mmaietta commented 9 months ago

Oh. In that case, just use this for now config.isNest = process.platform === 'win32' ? : config.isNest : false;

I don't know the impact to the production distributable though with isNest always false for docker images, but it does seem that the files don't have any previous signatures to "nest" on top of when building via Wine

mmaietta commented 9 months ago

If ☝️ works for signing on Docker Wine (you'll have to test the production distributable locally on your own OS versions you normally test on), then I think I fixed it: https://github.com/electron-userland/electron-builder/pull/8023/files#diff-d528668edd92176bca81f46dbf20bb548360fb3ba7c1cd5e79e468541b44a39fR66

Nantris commented 9 months ago

Fwiw I finally got a prototype Docker build container working and signing was not an issue for us with NSIS. We don't currently build .appx so I can't report on that.

We built using electronuserland/builder:18-wine running from a Windows 11 host.

Nantris commented 9 months ago

I realize I was unclear - we did not have to make any changes to anything to get codesign working.

I'm not certain of the electron-builder version we used or if it matters but if anyone is curious I can report back.

mmaietta commented 9 months ago

Wait, can you confirm whether we need this? https://github.com/electron-userland/electron-builder/pull/8023/files#diff-d528668edd92176bca81f46dbf20bb548360fb3ba7c1cd5e79e468541b44a39f

I still saw -nest failing on docker wine in the unit tests, so I'm curious as to what changed in your environment for it to now work.

Nantris commented 9 months ago

Since those are unmerged I assume that my success didn't rely on them.

Just to note, I'm not OP and he may still face the issue. I only just happened to start experimenting with Docker builds at the time this issue was filed.

I can confirm you that codesign worked for us with the electronuserland/builder:18-wine image running electron-builder@24.6.4 and no additional changes were necessary. But this was not in the context of a CI system.


Below are just some stabs in the dark:

I ask because of this:

I just set it up locally and it also won't work bare on macOS

@idoodler does your electron-builder.env file looks exactly like this?

CSC_KEY_PASSWORD=123456

I wonder if there are multiple valid key file extensions? We used .pfx. And is there any chance that you converted the key file incorrectly from another format?

idoodler commented 9 months ago

@Slapbox I will try to verify it by changing electronuserland/builder:wine to electronuserland/builder:18-wine

Regarding the electron-builder.env

It obviously doesn't look exactly like this, I do have the following keys defined WIN_CSC_KEY_PASSWORD, APPLE_API_KEY, APPLE_API_KEY_ID, APPLE_API_KEY_ISSUER, we use a .p12 format, as we did in our currently running project.

The currently running project uses a very old version electron-builder and is build on a macMini with Paralels Desktop.

idoodler commented 8 months ago

I just tried it with electronuserland/builder:18-wine this results in /root/.cache/electron-builder/nsis/nsis-3.0.4.1/linux/makensis process failed ERR_ELECTRON_BUILDER_CANNOT_EXECUTE Exit code: 84

idoodler commented 8 months ago

Could the package name be an issue? The name includes an organization, like @myorg/app -> /builds/App/monorepo/apps/raptor-electron/dist/@myorgapp-1.0.0-x64.nsis.7z.

I trued to change that setting a productName in the electron-builder.yml file but its still the same path

cZalyun commented 8 months ago

I had the same error with 24.12.0 (NOT THE DOCKERIZED VERSION) reverted to 24.9.1 it works perfectly again. (building/signing on a 9 years old mac mini) node: v20.9.0 electron: v29.0.1

idoodler commented 8 months ago

@cZalyun I just pinned electron-builder to 24.12.0 without any change.

beaugunderson commented 8 months ago

We just upgraded from 24.9.1 to 24.12.0 and see an identical error. We do not use docker and are building on latest macOS.

Reverting to 24.9.1 does fix the issue for us.

mmaietta commented 8 months ago

Okay, will investigate this asap.

Can someone provide me a minimum reproducible repo in case I can't reproduce locally?

Related note, if I provide a patch-package patch, would any of you be willing to try it out locally to confirm whether it fixes the issue?

beaugunderson commented 8 months ago

I am willing to try a patch-package patch 🙏

My .env looks like:

WIN_CSC_LINK=./beau-gunderson.pfx
WIN_CSC_KEY_PASSWORD=<redacted>

And the relevant part of my electron-builder.json5 looks like:

  "win": {
    "target": [
      {
        "target": "nsis",
        "arch": [
          "x64"
        ]
      }
    ],
    "artifactName": "${productName}_${version}.${ext}"
  },
  "nsis": {
    "oneClick": false,
    "perMachine": false,
    "allowToChangeInstallationDirectory": true,
    "deleteAppDataOnUninstall": false
  }
idoodler commented 8 months ago

I'd also be willing to test any patches. Locally I am able to build it tho. The issue only appears on our buildserver when utilizing the docker image

idoodler commented 8 months ago

As a workaround I am now building NSIS on a Mac Studio M2 Max, our new macOS buildserver. Tho I had to use electron-builder in version 24.9.1 to also work around the issue mentioned previously.

mmaietta commented 8 months ago

Actually, you may be able to test this without a patch-package using a win-sign.js hook to test this approach:

const { doSign } = require('app-builder-lib/out/codeSign/windowsCodeSign')

/**
 * @type {import("electron-builder").CustomWindowsSign} sign
 */
module.exports = async function sign(config, packager) {
  config.isNest = process.platform === 'win32' ? : config.isNest : false;
  await doSign(config, packager)
}

Add to win.sign configuration property. I'm not fully familiar with what -nest is doing, but my impression is that it's trying to append a signature to a file, which doesn't exist within a docker image? You'll need to test this from your side to make sure no side-effects

Anyone willing to give it a shot?

I'll continue reviewing the (massive) diff between the versions to see if anything sticks out to me

beaugunderson commented 8 months ago

It doesn't appear that changing the config there is respected...

  • loaded configuration  file=/Users/beau/p/meris-editor/electron-builder.json5
  • writing effective config  file=release/1.0.6/builder-effective-config.yaml
  • rebuilding native dependencies  dependencies=@julusian/midi@3.2.0 platform=win32 arch=x64
  • rebuilding native dependency  name=@julusian/midi version=3.2.0
  • packaging       platform=win32 arch=x64 electron=28.2.3 appOutDir=release/1.0.6/win-unpacked
  • signing         file=release/1.0.6/win-unpacked/Meris prEDITOR.exe certificateFile=/Users/beau/p/meris-editor/beau-gunderson.pfx
{ platform: 'darwin', isNest: false }
{ platform: 'darwin', isNest: false }
  ⨯ Exit code: 255. Command failed: /Users/beau/Library/Caches/electron-builder/winCodeSign/winCodeSign-2.6.0/darwin/10.12/osslsigncode -in /Users/beau/p/meris-editor/release/1.0.6/win-unpacked/Meris prEDITOR.exe -out /Users/beau/p/meris-editor/release/1.0.6/win-unpacked/Meris prEDITOR-signed-sha256.exe -t http://timestamp.digicert.com -pkcs12 /Users/beau/p/meris-editor/beau-gunderson.pfx -h sha256 -n Meris prEDITOR -i https://github.com/beaugunderson/meris-editor -nest -pass acdb1c392c6e5213165aa2d57bdf655f19ee1d8cb3b9e4129a6086cd97b06016 (sha256 hash) to extract existing signature in -nest mode
Failed

Unable to extract existing signature in -nest mode

I used a modified version that prints the values:

module.exports = async function sign(config, packager) {
  config.isNest = process.platform === 'win32' ? config.isNest : false;
  console.log({ platform: process.platform, isNest: config.isNest });
  await doSign(config, packager)
}
mmaietta commented 8 months ago

Thanks for checking! I'll continue my debugging

mmaietta commented 8 months ago

Fixed in v24.13.3 Apologies for any frustrations this caused and thanks for your patience!

idoodler commented 8 months ago

Thanks @mmaietta, 24.13.3 fixes the issue on macOS. It is however still broken when using electronuserland/builder:wine. In addition when building mac dmg I am getting the following error: ⨯ Env vars APPLE_API_KEY, APPLE_API_KEY_ID and APPLE_API_ISSUER need to be set. All these variables are defined in the electron-builder.env file unchanged and it also works with 24.9.1.

It fails with: 84 failedTask=build stackTrace=Error: /root/.cache/electron-builder/nsis/nsis-3.0.4.1/linux/makensis process failed ERR_ELECTRON_BUILDER_CANNOT_EXECUTE

mmaietta commented 8 months ago

Hmmm, I'll check docker wine.

For the mac dmg, please create a separate issue and post logs with DEBUG=electron-builder and your notarization config

mmaietta commented 8 months ago

Update: I can't test or reproduce on mac arm64 because electron-builder-binaries asset rcedit is an ia32 binary, so it isn't executable within the docker image due to mac arm64 only able to emulate x64 binaries.

I'll need to see if I can get it running on a Windows machine, since even a Windows VM won't work on my laptop.

Running within the wine docker image on the GH CI returns this error, which doesn't align with your -nest error the OP mentions

 Failed

    Unrecognized file type: /tmp/et-db411cd21f3bbd882f3a5a30c58db6ff/t-VB1Fns/test-project-0/dist/win-unpacked/resources/app.asar.unpacked/assets/nested/nested/file.exe

@idoodler , what was the last electron-builder version and/or docker image this was working for you?

idoodler commented 8 months ago

@mmaietta Currently I am running the following config: electron-builder -> 24.9.1

I also encountered that issue with rcedit, thats why we bought the Intel machine.

mmaietta commented 8 months ago

Thanks for the additional info. Was there a previous version that this was working on? I'm trying to git bisect/diff changes to identify how long this issue has been present.

The fact that I can't get the Github CI to trigger the same error worries me as that makes it even harder to debug/reproduce or even write a unit test for.

idoodler commented 8 months ago

@mmaietta With that project I wasn't able to build NSIS on Docker with a certificate. I always got that error message. "Bare Mettal" on a macOS machine works however.

At firt I thought it is because we only have Apple ARM Macs here in the company, thats why we bought the Intel Machine we then set up as a Gitlab docker runner.

Do you need any logs I may be able to find some time to prepare a minimal project where the issue occures.

mmaietta commented 8 months ago

Yes, a minimal project would be immensely helpful! 🙂

I'd also really like to know if this was working previously in an earlier version of electron-builder, something I can render a git diff with as my only intel machine is my Windows gaming rig