Windows signatures failing: SignerSign() failed." (-2147024671/0x800700e1)

[jakubgs]

Since last week we've been seeing failures to sign DLLs and EXE files in CI for Windows builds:

+ signtool.exe sign -debug -fd SHA256 -p **** -f **** -tr http://timestamp.digicert.com ./tmp/windows/dist/Status/bin/Status.exe
SignTool Error: An unexpected internal error has occurred.

After EKU filter, 1 certs were left.
After expiry filter, 1 certs were left.
After Private Key filter, 1 certs were left.
The following certificate was selected:
    Issued to: status.im
    Issued by: status.im
    Expires:   Mon May 04 10:34:53 2122
    SHA1 hash: 4BE16A13BE2CB4282D0C70BA61FE0BA3B7D0CFFE

The following additional certificates will be attached:
Done Adding Additional Store
Error information: "Error: SignerSign() failed." (-2147024671/0x800700e1)

https://ci.status.im/job/status-desktop/job/prs/job/windows/job/x86_64/job/package/job/PR-16189/5/console

[jakubgs]

I have attempted to bisect code changes to find the issue, but today all builds are succeeding:

https://ci.infra.status.im/job/status-desktop/job/systems/job/windows/job/x86_64/job/bisect/

[jakubgs]

This led me to believe that the issue might be with something we do not control, which is the timestamp server:

signtool.exe sign -debug -fd SHA256 -p **** -f **** -tr http://timestamp.digicert.com ${BINARY_TO_SIGN}

For this reason I've contacted DigiCert support about possible issues with their server.

[jakubgs]

After talking to their support and mentioning our issues I got this response:

Aah I see, I have picked one or 2 customers reaching out regarding our time stamp

One managed to refresh and get it up
And some tested with different servers
So there was no confirmed issues with the server entirely

When experiencing these drops, can you kindly test with the below timestamp servers as well,
to check and confirm if its really the timestamp server

So it is possible that was the issue. I was also given a list of alternative timestamp servers we can try:

http://timestamp.sectigo.com/
http://timestamp.globalsign.com/tsa/r6advanced1
http://rfc3161timestamp.globalsign.com/advanced
http://timestamp.apple.com/ts01
http://tsa.mesign.com/
http://time.certum.pl/
https://freetsa.org/

[jakubgs]

Apparently we can also check their service status at: https://status.digicert.com/

You are most welcome,

Yes, when our timestamp server is down, we update our status page, and this is updated by reported issues and running some tests to the server

https://status.digicert.com/

But currently it shows no issues.

[jakubgs]

My attempts to debug the issue have resulted in some changes:

• https://github.com/status-im/status-desktop/pull/16189

But it appears -debug flag for signtool.exe sign provides no additional useful input in this case.

[jakubgs]

Apparently signtool.exe should create logs under: C:\Users\jenkins\Signing Manager\logs

Can you kindly check the signtool logs and share

Below is where you can get your Logs:

C Drive >> Users >> Select your user >> Signing Manager >> Then logs

But no such folder exists for jenkins.

[jakubgs]

Apparently another possibility is Windows Defender flagging the binary file as malicious:

I have checked and confirmed with my Senior, after running some further checks on the file and the error.

It looks like the windows defender is flagging the file as a malware, therefore preventing the the file from getting signed.

So this is more of a file issue than the timestamp or the certificate.

Below is one of the sources we have managed to get further information on this error.

Please note that this is not a DigiCert source (checking the "Re: Error Signing an exe-file Quote Tue Oct 24, 2023 7:36 am"):

https://www.advancedinstaller.com/forums/viewtopic.php?t=51721

And indeed:

Unfortunately these seems to be false positive detections of Windows Defender. The quickest action you can implement will be to add a Defender exclusion on the output folder where your setup project is built.

• https://www.advancedinstaller.com/forums/viewtopic.php?t=51721

[jakubgs]

It's quite easy to add an exclusion rule to Defender settings via UI:

But we need a solution that can be applied with Ansible or Powershell in an automated manner.

[jakubgs]

It appears to be possible with Add-MpPreference PowerShell command: https://learn.microsoft.com/en-us/powershell/module/defender/add-mppreference?view=windowsserver2022-ps

[jakubgs]

I have added exclusion of workspace folder in infra-ci:

• https://github.com/status-im/infra-ci/commit/9fb0f567 - jenkins-slave-user: exclude workspace dir from Defender

[jakubgs]

Currently we are still seeing some signing errors, but the error code is different:

21:06:12  Error information: "Error: SignerSign() failed." (-2146869243/0x80096005)

https://ci.status.im/job/status-desktop/job/systems/job/windows/job/x86_64/job/package/799/console

And I did try a different timestamp server from Sectigo.

[jakubgs]

It appears to be also a timestamp error, but due to wrong hashing algorithm being used:

• https://stackoverflow.com/questions/62161135/signersign-failed-2146869243-0x80096005
• https://github.com/electron-userland/electron-builder/issues/5166#issuecomment-663568037

Apparently the Sectigo URL can be used but needs an argument:

http://timestamp.sectigo.com/?td=sha256

[jakubgs]

Fix for the URL:

• https://github.com/status-im/status-desktop/pull/16215

[jakubgs]

I'm going to close this, we can always reopen if it happens again.