AIR bundle signing failure - Packaging failed: Could not generate timestamp: Connection reset

philpme commented 1 month ago

Problem Description

Sometime in the past week both our window and mac native bundle builds have started failing with the error

Packaging failed: Could not generate timestamp: Connection reset

Steps to Reproduce

With AIR 51.1.0.3, attempt a bundle build - such as for mac

./bin/adt -package -storetype KeychainStore -alias "Developer ID Application: XXX (XXX)" -target bundle -storetype KeychainStore -alias "XXX (XXX)" -tsa http://timestamp.digicert.com ...

returns the error Packaging failed: Could not generate timestamp: Connection reset

Known Workarounds

add the following tho the hosts file on the machine to redirect access of sha256timestamp.ws.symantec.com to timestamp.digicert.com

216.168.244.9 sha256timestamp.ws.symantec.com

The above command now works even though our we didn't reference sha256timestamp.ws.symantec.com

Seems to be related to this messaging from Digicert - however they were a few weeks late in shutting down the service, as we were successfully able to build last week

The timestamp url in the answer will no longer work after 7/24/2024 The latests timestamp URL is: timestamp.digicert.com 216.168.244.9 Note Explaining: On July 24, 2024, at 17:00 MDT (23:00 UTC) DigiCert will shut down our legacy Symantec timestamping service. If you or customers use timestamping when signing executables or documents, you may need to change the timestamp URL in your signing tool to the newer DigiCert service, timestamp.digicert.com, before the shutdown occurs.

ajwfrost commented 1 month ago

Just checking, what happens if you take away your hosts workaround and instead add the -tsa argument to the first set of signing options too? i.e.

./bin/adt -package -storetype KeychainStore -alias "Developer ID Application: XXX (XXX)" -tsa http://timestamp.digicert.com -target bundle -storetype KeychainStore -alias "XXX (XXX)" -tsa http://timestamp.digicert.com ...

It does look we have a mixture of timestamp server usage in ADT, for some reason - so some code paths or platforms might end up with this Symantec URL being the default for signing.. which we need to change!!

thanks

philpme commented 1 month ago

Ah yes, adding the -tsa entry for both sets of signing args also work. It seems adt has sha256timestamp.ws.symantec.com as a default if it's not supplied

RossD20Studios commented 2 weeks ago

Hi @philpme @ajwfrost - I have tried your solution to update the host file on my Mac using steps here: https://kinsta.com/knowledgebase/edit-mac-hosts-file/

Neither 216.168.244.9 sha256timestamp.ws.symantec.com or 216.168.244.9 timestamp.digicert.com works.

I've also noticed that attempting to build with Adobe Animate and setting the timestamp URL to http://timestamp.digicert.com or http://sha256timestamp.ws.symantec.com/sha256/timestamp also fails.

I had previously used http://sha256timestamp.ws.symantec.com/sha256/timestamp with Animate, but this no longer works.

What is the correct timestamp URL that will work?

RossD20Studios commented 2 weeks ago

@ajwfrost - If I'm packaging with IntelliJ, where/how can I specify the "-tsa http://timestamp.digicert.com" argument?

ajwfrost commented 2 weeks ago

Hi I just checked that 216.168.244.9 is still the correct thing. It should be the first one you mention there, you don't want/need to redirect the digicert address:

216.168.244.9 sha256timestamp.ws.symantec.com

The URL that works is what you'd then mentioned you're trying in IntelliJ:

-tsa http://timestamp.digicert.com/

Worth checking that you can reach this from your network perhaps? The test for it is:

 curl -vi timestamp.digicert.com/timestamp/health/heartbeat

and if your hosts file is correctly set up, it would be the exact same result if you tried:

 curl -vi sha256timestamp.ws.symantec.com/timestamp/health/heartbeat

(including the same IPv4 address resolution at the start of the output).

In my version of IntelliJ:

In the menu system: File -> Project Structure...
On the left of the dialog: Project Settings > Modules
Then on the top right: AIR Package
At the bottom of this are the signing options - check the "More options" box
In the bottom-right "TSA" edit field, add http://timestamp.digicert.com

RossD20Studios commented 2 weeks ago

Hi @ajwfrost -

I reset my host file back:

I modified the TSA field in IntelliJ:

I tested my connections:

Timestamp.digicert.com appears to connect, but shad256timestamp might have an error? Not sure what to make of the console output above.

In either case, after following these steps, I still get the same error:

What's my next step to correct this error?

RossD20Studios commented 2 weeks ago

Scratch that @ajwfrost - I've got it working! Thinking that the issue might be my ISP, I tried switching over to my cell phone hotspot and it worked! Then I switched back, and it worked on my regular connection. So, probably just some kind of cache that didn't resolve until the connection reset (even though I did run the -HUP mDNSResponder console command that was supposed to do that).

So, in case anyone else has this happen, try disconnecting/reconnecting your internet :)

ajwfrost commented 2 weeks ago

Hi

That all looks good - you'd expect the error from the symantec version since that's the one that got retired. I would be interested in that "ADT command line" link though, to see what it's actually calling. Or you could check the ~/adt.log file if that's being generated (or if not, use the AIR SDK Manager's "Troubleshooting" tab...)

When I added that TSA entry, IntelliJ then correctly called ADT with the requested timestamp URL and it all connected fine, so I'm not sure what's happening there.

But you could also add that hosts entry .. after which, it might be that the curl command as well as AIR packaging should start working again with the symantec url...

216.168.244.9   sha256timestamp.ws.symantec.com

thanks

ajwfrost commented 2 weeks ago

Ah - caches! Hadn't even thought of that .... :-)

thanks

RossD20Studios commented 1 week ago

Hey @ajwfrost - Sorry to report that I'm experiencing the timestamp server again after updating to the latest version of AIR, v51.4. It's failing to timestamp in both IntelliJ and in Adobe Animate using the http://timestamp.digicert.com/ URL

RossD20Studios commented 1 week ago

Update: Ok, it seems like the digicert service itself is down according to: https://downforeveryoneorjustme.com/timestamp.digicert.com

Is there a backup timestamp URL we can use?

ajwfrost commented 1 week ago

I'm not sure that this down-detector is able to properly check the timestamp server, as they do return some error codes if you try to access this via a web browser... that site seems to suggest it's still down, but I'm able to timestamp some .air files with it at the moment...

Some others: http://tsa.mesign.com http://rfc3161.ai.moda http://ts.ssl.com/

There are more available but for some of them, ADT says that the response is invalid, which may just mean that we need to update it to be able to cope with a wider range of responses.. but hopefully this list would be sufficient unless anyone has a strong need to use another one...

thanks

RossD20Studios commented 6 days ago

Thanks @ajwfrost - timestamp.digicert.com is still not working for me (and it's still reported as down), but I was able to sign with tsa.mesign.com :)

airsdk / Adobe-Runtime-Support