search

eclipse-simrel / simrel.build

The aggregation model and build infrastructure.
3 stars 41 forks source link

Content signed with an expired certifcate after the expiration date is considered unsigned #363

Closed merks closed 3 months ago

merks commented 4 months ago

I will edit this description with additional details as they become available.

This issue affects all projects that sign jars, not just SimRel projects.

This issue directly related to the problem is already opened:

https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/issues/4662

Here is a (hopefully correct) short list of affected projects:

The signing certificate expired May 21, 2024:

image

Any content signed with this certificate after that date is considered unsigned, e.g., 

$/c/Program\ Files/Java/jdk-21.0.2+13/bin/jarsigner.exe  -verbose -verify /d/stuff/org.eclipse.e4.ui.dialogs_1.5.0.v20240424-0957.jar

s       4608 Wed Apr 24 22:09:26 CEST 2024 META-INF/MANIFEST.MF
        3961 Wed Apr 24 22:09:26 CEST 2024 META-INF/ECLIPSE_.SF
        9554 Wed Apr 24 22:09:26 CEST 2024 META-INF/ECLIPSE_.RSA
           0 Wed Apr 24 22:09:24 CEST 2024 META-INF/
           0 Wed Apr 24 22:09:24 CEST 2024 org/
           0 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/
           0 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/
           0 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/
           0 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/dialogs/
           0 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/dialogs/filteredtree/
           0 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/dialogs/textbundles/
           0 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/internal/
           0 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/internal/dialogs/
           0 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/internal/dialogs/about/
           0 Wed Apr 24 22:09:24 CEST 2024 icons/
           0 Wed Apr 24 22:09:24 CEST 2024 icons/full/
           0 Wed Apr 24 22:09:24 CEST 2024 icons/full/dtool16/
           0 Wed Apr 24 22:09:24 CEST 2024 icons/full/etool16/
sm      2520 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/dialogs/filteredtree/BasicUIJob.class
sm     13542 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/dialogs/filteredtree/FilteredTree.class
sm      3362 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/dialogs/filteredtree/FilteredTree$NotifyingTreeViewer.class
sm      1397 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/dialogs/filteredtree/FilteredTree$5.class
sm      1199 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/dialogs/filteredtree/FilteredTree$4.class
sm      2604 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/dialogs/filteredtree/FilteredTree$2.class
sm      1270 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/dialogs/filteredtree/FilteredTree$3.class
sm      4743 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/dialogs/filteredtree/FilteredTree$1.class
sm      7594 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/dialogs/filteredtree/PatternFilter.class
sm      1258 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/dialogs/textbundles/E4DialogMessages.class
sm      1663 Wed Apr 24 22:03:14 CEST 2024 org/eclipse/e4/ui/dialogs/textbundles/messages.properties
sm      9637 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/internal/dialogs/about/AboutDialogE4.class
sm      1838 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/internal/dialogs/about/AboutText$1.class
sm     11033 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/internal/dialogs/about/AboutText.class
sm      2222 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/internal/dialogs/about/AboutText$2.class
sm      4111 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/internal/dialogs/about/BrandingProperties.class
sm      2556 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/internal/dialogs/about/HyperlinkExtractor.class
sm       719 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/internal/dialogs/about/HyperlinkRange.class
sm       421 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/internal/dialogs/about/IProductConstants.class
sm      1911 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/internal/dialogs/about/ParsedAbout.class
sm      2617 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/internal/dialogs/about/ProductInformation.class
sm      7441 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/internal/dialogs/about/ProductProperties.class
sm      1084 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/internal/dialogs/about/UnavailableProduct.class
sm       397 Wed Apr 24 22:03:14 CEST 2024 icons/full/dtool16/clear_co.png
sm       755 Wed Apr 24 22:03:14 CEST 2024 icons/full/dtool16/clear_co@2x.png
sm       463 Wed Apr 24 22:03:14 CEST 2024 icons/full/etool16/clear_co.png
sm      1015 Wed Apr 24 22:03:14 CEST 2024 icons/full/etool16/clear_co@2x.png
sm       214 Wed Apr 24 22:09:24 CEST 2024 .api_description
sm      1460 Wed Apr 24 22:03:14 CEST 2024 about.html
sm       614 Wed Apr 24 22:03:14 CEST 2024 plugin.properties

  s = signature was verified
  m = entry is listed in manifest
  k = at least one certificate was found in keystore

- Signed by "EMAILADDRESS=webmaster@eclipse.org, CN="Eclipse.org Foundation, Inc.", OU=IT, O="Eclipse.org Foundation, Inc.", L=Ottawa, ST=Ontario, C=CA"
    Digest algorithm: SHA-256
    Signature algorithm: SHA384withRSA, 4096-bit key
  Timestamped by "CN=Symantec SHA256 TimeStamping Signer - G3, OU=Symantec Trust Network, O=Symantec Corporation, C=US" on Mi. Apr. 24 22:09:27 UTC 2024
    Timestamp digest algorithm: SHA-256
    Timestamp signature algorithm: SHA256withRSA, 2048-bit key

jar verified.

Warning:
This jar contains entries whose TSA certificate chain is invalid. Reason: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
POSIX file permission and/or symlink attributes detected. These attributes are ignored when signing and are not protected by the signature.

Re-run with the -verbose and -certs options for more details.

The signer certificate expired on 2024-05-22. However, the JAR will be valid until the timestamp expires on 2029-03-23.

We see SimRel staging is badly affected by this:

image

Note that some shown with strikeout and some are not. That's because some artifacts were signed by the certificate when it was still valid while others are signed by the certificate after is expired.

Here are JUnit-style test results:

https://ci.eclipse.org/simrel/job/simrel.oomph.repository-analyzer.test/lastCompletedBuild/testReport/

cdietrich commented 4 months ago

is there already a new cert available?

Kellindil commented 4 months ago

Hello,

I will assume this means we will have to rebuild all projects currently contributed to the simrel, even those which didn't plan new versions?

Laurent

merks commented 4 months ago

@cdietrich

No, I believe the new cert is not yet available.

@Kellindil

No, not all projects need to rebuild. It's perfectly find to have a jar signed with an expired signature as long as the siganature itself was produced within the certificates validity range. This is why I mentioned that some artifacts are in strikeout but some are not. I will collect a list of affected projects ASAP.

Please look for your artifacts in this list:

https://ci.eclipse.org/simrel/job/simrel.oomph.repository-analyzer.test/lastCompletedBuild/testReport/

merks commented 4 months ago

This is the impact on the users:

image

mbarbero commented 4 months ago

The certificate expired yesterday, so anything built and signed before should be valid.

There is ongoing work to update the certificate. However, this update is not routine because the secret key must now be stored on an HSM, a new requirement from the code-signing CA.

https://www.eclipsestatus.io/incident/373129

iloveeclipse commented 4 months ago

Please look for your artifacts in this list:

https://ci.eclipse.org/simrel/job/simrel.oomph.repository-analyzer.test/lastCompletedBuild/testReport/

It does not include org.eclipse.e4.ui.dialogsmentioned in your error above. Can we get a report for the platform somewhere?

merks commented 4 months ago

@iloveeclipse

I think you already noticed that I provided the platform-specific report here:

The Platform's M3 contribution was last Friday, so that content is not a problem; the Platform's own pend RC1 candidate content is a problem.

akurtakov commented 4 months ago

I wonder is it possible for simrel verification to fail if contributed artifacts contain bad signature?

pcdavid commented 4 months ago

Shouldn't signing itself fail if the certificate is expired?

Bralic commented 4 months ago

If possible, of course, that'd be nice.

merks commented 4 months ago

@akurtakov

Right now the verification only uses the p2 planner to ensure there is a solution. Checking all the artifact signatures could be done by the aggregation step, which takes much longer. Right now actual testing of signatures is done by Oomph's repository analyzer as done by this job:

https://ci.eclipse.org/simrel/job/simrel.oomph.repository-analyzer.test/

Unfortunately that didn't send me an email because it didn't actually fail. 😞

The Platform's content verification sent me an email this morning so that's what I noticed first. Of course I check the report before promoting the content so that's what I noticed second.

Several factors contributed to this problem. Proactively updating the certificates would prevent all these problems. The signer verifying that the artifact is really properly signed would help catch that oversight when that proactive update hasn't taken place. Projects using the Oomph analyzer themselves would also catch this problem. And finally my complacency also contributed. That's because the aggregation builds PGP signs everything that is unsigned thanks to Tycho's support for PGP signing with skipIfJarsignedAndAnchored. So I generally never expect to see a signing failures for SimRel because that feature always kicks in. Unfortunately that Tycho feature does not call org.eclipse.osgi.signedcontent.SignedContent.checkValidity(SignerInfo) as part of signature checking, so it did not notice that this content, while signed, is not actually considered signed because it's not signed validly. Blah, blah, blah. I'm working to fix that Tycho problem and @laeubi already planned a new Tycho release for tomorrow, so I'm trying to get that in ASAP.

Bralic commented 4 months ago

Thank you for all this information! Very instructive! And of course all the effort.

merks commented 4 months ago

FYI, I added this content to the start of this issue:

Here is a (hopefully correct) short list of affected projects:

iloveeclipse commented 4 months ago

Here is a (hopefully correct) short list of affected projects:

What about Platform?

merks commented 4 months ago

@iloveeclipse

I answered that here: 😱

https://github.com/eclipse-simrel/simrel.build/issues/363#issuecomment-2126501998

iloveeclipse commented 4 months ago

I answered that here: 😱

#363 (comment)

OK, so this ticket is for all projects except platform?

merks commented 4 months ago

The Platform's own build is affected, i.e, the current RC1 candidate has bad signatures. In fact all projects using the CBI jar signing service yesterday and today are affected. The intent of the list is for the SimRel participants who will need to rebuild yet another new contribution once the new signing certificate is available for the CBI jar signing service. As such, the Platform will need to respin RC1 before contributing that to SimRel on Friday as currently scheduled.

ruspl-afed commented 4 months ago

who will need to rebuild yet another new contribution once the new signing certificate is available for the CBI jar signing service.

Do you plan to provide cross-project notification so that we all know that we can start rebuilding Ed @merks ?

merks commented 4 months ago

Yes, I will break my own rule and send an email when folks can respin. I've seen no updates here:

https://www.eclipsestatus.io/incident/373129

As you've probably seen on cross-projects, M3 is canceled so no one needs to lose sleep or hair over this.

merks commented 4 months ago

FYI, there is currently no ETA for the incident:

https://www.eclipsestatus.io/incident/373129

netomi commented 4 months ago

so the jar signing service is operational again.

We are still working on bringing back the authenticode signing service for windows executables, but it would be great if people can already test the jar signing service and report if everything works as expected.

jonahgraham commented 4 months ago

it would be great if people can already test the jar signing service and report if everything works as expected.

I kicked off a CDT build https://ci.eclipse.org/cdt/job/cdt/job/main/400/ - but if the windows signing is still offline the build will probably fail as CDT contains a handful of dlls that need to be signed.

jonahgraham commented 4 months ago

The build is running still (take a long time) but jar signing is happening and windows signing is running without error. I'll try to find time to test and report back once the build is finished later today

cdietrich commented 4 months ago 
/Library/Java/JavaVirtualMachines/temurin-21.jdk/Contents/Home/bin/jarsigner --verify org.eclipse.emf.mwe2.language.ide_2.18.0.20240524-1707.jar

JAR-Datei verifiziert.

=> looks good

netomi commented 4 months ago

the windows / authenticode service has also been redeployed and should be working again, please test and report.

merks commented 4 months ago

Thanks @netomi and the IT team!

I will close this issue after I've verified that the staging repository no longer contains content signed with the expired certificate.

LorenzoBettini commented 4 months ago

EMF Parsley should also be fixed now.

Godin commented 4 months ago

Prior to https://github.com/eclipse-simrel/simrel.build/pull/369

jarsigner -verify -verbose org.eclipse.eclemma.core_3.1.9.202405220136.jar | grep expire

This jar contains entries whose signer certificate has expired.
The signer certificate expired on 2024-05-22. However, the JAR will be valid until the timestamp expires on 2029-03-23.

and after

jarsigner -verify -verbose org.eclipse.eclemma.core_3.1.9.202405260028.jar | grep expire

The signer certificate will expire on 2026-06-12.
The timestamp will expire on 2031-11-10.

so EclEmma should also be fixed now.

msohn commented 3 months ago

I contributed a new build for JGit/EGit: #385

merks commented 3 months ago

FYI, I have a little analyzer hacked tougher to read the test output and then produce output like this:

image

I compare that to the state of the checked items at the start of this issue so I can easily and accurately track the process.

I also see all PRs being created and merged, so I know when to check content again. 😁

titou10titou10 commented 2 months ago

It seems this is not fully solved for eclipse RCP 2024-06

The certificate associated to "eclipse.exe" RCP v2024-06 for windows,downloaded from eclipse.org is ok , expiring in 2026-06-21

But when we build an app with tycho v4.0.8 and the tycho-p2-director-pluginplugin, the generated windows executable created has a certificate that expired 2024-05-21 see https://github.com/jmstoolbox/jmstoolbox/issues/173

Is there a workaround to this problem? thanks

jonahgraham commented 2 months ago

@titou10titou10 I think you are seeing something slightly different. The eclipse.exe discussed in https://github.com/jmstoolbox/jmstoolbox/issues/173 was signed with a valid certificate, and the screenshots show that that the digital signature is ok. I'm not sure what we can do about that unfortunately. Can you ask your user to share the details of the error they are seeing? I'll follow on your issue to see if I can help and if we find something generally useful we can report it back here.

titou10titou10 commented 2 months ago

The eclipse.exe discussed in jmstoolbox/jmstoolbox#173 was signed with a valid certificate, and the screenshots show that that the digital signature is ok.

@jonahgraham I'm not sure to understand your point When I build JMSToolBox with tycho ("classic" build for eclipse RCP apps), the signature of the generated exe is not valid When you right click on the generated executable > Digital Signature Information, the windows that appears clearly state that "the digital signature is not valid"

It is the same if I build, for example DBeaver, another RCP based app (https://github.com/dbeaver/dbeaver)

(sorry for the bad quality of the picture) The printscreen for JMSToolBox freshly build this morning with tycho 4.08 and eclipse RCP 2024-06

Sans titre

To be clear, in french: "Cette signature numérique n'est pas valide" = This digital signature is not valid "La signature numérique de l'objet n'a pas pu se verifier" = The digital signature could not be verified "Valide du/au" = Valid from/to

jonahgraham commented 2 months ago

I don't understand why the screenshot in https://github.com/jmstoolbox/jmstoolbox/issues/173#issue-2375737601 is showing valid, but your screenshot is showing invalid. As @mbarbero said in https://github.com/jmstoolbox/jmstoolbox/issues/173#issuecomment-2203497356 the thing that matters is when it was signed, not today's date. Your screenshot shows it was signed on 7 May, and that is within the validity dates.

When I check the launcher I see the same file size as you, and Windows reports certs as valid. Here is what I checked:

$ sha512sum $PWD/*
6b1a7cb2faee77804b889797793833a4c80dc6d3a0f04f43936e93c174f7c355e149a53a32c410b2d7c1b9638d9b9aeb5e2113402cc8142fdd9d4d3242ff47f9  /tmp/a/jmstoolbox/org.titou10.jtb.product/target/org.eclipse.equinox.executable-3.8.2500.v20240513-1750/bin/win32/win32/x86_64/eclipsec.exe
1a099b90844a51065687ea3c352732649b8cdb1a34647d77c8c758970a6d694d86aeaa88442a811ee123a81b835b8b1dbb597067fe1a0a0981689d464fce2a64  /tmp/a/jmstoolbox/org.titou10.jtb.product/target/org.eclipse.equinox.executable-3.8.2500.v20240513-1750/bin/win32/win32/x86_64/launcher.exe

Can you check that nothing in your build process is modifying anything after the executables are extracted?

titou10titou10 commented 2 months ago

Honestly the build process is very straight forward:

Maybe it is tycho that is modifying the final exe? I must say, I'm far from an expert in tycho/maven etc...

If you download the artefacts from the action build from github, the JMSToolBox.exe has the same "problem", ie "The digital signature is not valid" : https://github.com/jmstoolbox/jmstoolbox/actions/runs/9598536715

jonahgraham commented 2 months ago

(Conversation moved back to https://github.com/jmstoolbox/jmstoolbox/issues/173 because the issue is unrelated to #363)

netomi commented 2 months ago

I added some comment in the referenced ticket, I strongly believe the expired signing certificate is just a red herring, the signature of the file is invalid due to a digest mismatch due to the changed icon. This is unrelated to the problems we have been facing with the expired signing certificate to which this issue relates.