jetty / jetty.project

Eclipse Jetty® - Web Container & Clients - supports HTTP/2, HTTP/1.1, HTTP/1.0, websocket, servlets, and more
https://eclipse.dev/jetty
Other
3.83k stars 1.91k forks source link

Produce SBOM and deploy to Maven Central #9502

Closed joakime closed 9 months ago

joakime commented 1 year ago

Jetty version(s) Jetty 12

Enhancement Description We should produce an SBOM and have it be published to maven central somwhere.

Perhaps as an attached artifact on jetty-bom or as a new artifact jetty-sbom ?

joakime commented 1 year ago

https://snyk.io/blog/create-sboms-java-maven-gradle/

I don't know if an XML based SBOM or a SPDC based SBOM is better. Do we want to publish both?

@olamy do you know of any other projects that publish SBOMs to maven central?

olamy commented 1 year ago

I don't have strong opinions here. But what do we include in the sbom? As we are very modular some dependencies (such hazelcast, infinispan, jackson, maven etc...) are not per default part of the distribution but only "on demand". Publishing sbom including those dependencies could be wrong for users not using those modules.

joakime commented 1 year ago

I would say we publish anything we deploy to maven central on a groupId that starts with org.eclipse.jetty

sbordet commented 1 year ago

@joakime remind me why do we need a SBOM and how's different from the BOM?

joakime commented 1 year ago

Maven BOM is just maven coordinate space with versions.

SBOM has more information (in a condensed form), suitable for various tooling built around auditing of of dependencies (are you using the most current version? is the same version as the developer released? etc). CVE databases are starting to use these SBOMs to identify specific artifacts that are good/bad too.

This is an example of an XML form of SBOM ....

<component type="library" bom-ref="pkg:maven/com.fasterxml.jackson.core/jackson-databind@2.13.4?type=jar">
 <publisher>FasterXML</publisher>
 <group>com.fasterxml.jackson.core</group>
 <name>jackson-databind</name>
 <version>2.13.4</version>
 <description>General data-binding functionality for Jackson: works on core streaming API</description>
 <hashes>
   <hash alg="MD5">03cb7aea126610e4c96ca6d14d75cc55</hash>
   <hash alg="SHA-1">98b0edfa8e4084078f10b7b356c300ded4a71491</hash>
   <hash alg="SHA-256">c9faff420d9e2c7e1e4711dbeebec2506a32c9942027211c5c293d8d87807eb6</hash>
   <hash alg="SHA-512">23f32026b181c6c71efc7789a8420c7d5cbcfb15f7696657e75f9cbe3635d13a88634b5db3c344deb914b719d60e3a9bfc1b63fa23152394e1e70b8e7bcd2116</hash>
   <hash alg="SHA-384">e25e844575891b2f3bcb2fdc67ae9fadf54d2836052c9ea2c045f1375eaa97e4780cd6752bef0ebc658fa17400c55268</hash>
   <hash alg="SHA3-384">e6955877c2c27327f6814f06d681118be2ae1a36bc5ff2e84ad27f213203bf77c347ba18d9abc61d5f1c99b6e81f6c2d</hash>
   <hash alg="SHA3-256">88b12b0643a4791fa5cd0c5e30bc2631903870cf916c8a1b4198c856fd91e5f4</hash>
   <hash alg="SHA3-512">7e86a69bcf7b4c8a6949acce0ec15f33b74d5ac604f23cd631ec16bfdfd70d42499028b9d062648b31d7a187ea4dc98ec296a329f4cfd4952744ed1281fa9d9a</hash>
 </hashes>
 <licenses>
   <license>
     <id>Apache-2.0</id>
   </license>
 </licenses>
 <purl>pkg:maven/com.fasterxml.jackson.core/jackson-databind@2.13.4?type=jar</purl>
 <externalReferences>
   <reference type="vcs">
     <url>http://github.com/FasterXML/jackson-databind</url>
   </reference>
   <reference type="website">
     <url>http://fasterxml.com/</url>
   </reference>
   <reference type="distribution">
     <url>https://oss.sonatype.org/service/local/staging/deploy/maven2/</url>
   </reference>
 </externalReferences>
</component>

This is an example of an SPDX / SPDC form of SBOM ...

##### Package representing the jackson-databind

PackageName: jackson-databind
SPDXID: SPDXRef-Package-jackson-databind-2.13.4
PackageVersion: 2.13.4
PackageSupplier: Organization: jackson-databind
PackageDownloadLocation: https://mvnrepository.com/artifact/com.fasterxml.jackson.core/jackson-databind/2.13.4
FilesAnalyzed: false
PackageChecksum: SHA1: 7d03e73aa50d143b3ecbdea2c0c9e158e5ed8021
PackageHomePage: NOASSERTION
PackageLicenseConcluded: NOASSERTION
PackageLicenseDeclared: NOASSERTION
PackageCopyrightText: NOASSERTION
PackageLicenseComments: NOASSERTION
PackageComment: NOASSERTION

Relationship: SPDXRef-Package-jackson-databind-2.13.4 DEPENDS_ON SPDXRef-Package-jackson-annotations-2.13.4
Relationship: SPDXRef-Package-jackson-databind-2.13.4 DEPENDS_ON SPDXRef-Package-jackson-core-2.13.4
sbordet commented 1 year ago

All this information is already in Maven Central. Why do we need to duplicate it? Certainly tools can query Maven Central and build and XML with all the information out of it, from SHAs to GPG signatures, etc.

joakime commented 1 year ago

You'd think so, but no.

image

Some history here https://learn.sonatype.com/guides-old/sboms-explained/

And other tools to generate them here https://blog.sonatype.com/5-tools-to-automate-sbom-creation

riteshnoronha commented 1 year ago

The next question after publishing an SBOM is usually how consumable it is, each of the tool listed https://blog.sonatype.com/5-tools-to-automate-sbom-creation have different levels of maturity. We have created an open source sbom quality scoring tool, which can help with this process. https://github.com/interlynk-io/sbomqs

olamy commented 1 year ago

I would say we publish anything we deploy to maven central on a groupId that starts with org.eclipse.jetty

sounds good. I would go for https://github.com/CycloneDX/cyclonedx-maven-plugin As I understand it has some extra bonus with Maven Central and people maintaining it are usual suspects we can have easy access to ask bug fix or support :)

cc @hboutemy

hboutemy commented 1 year ago

yes, given your build is done with Maven, cyclonedx-maven-plugin looks like a good and easy solution you'll have one SBOM per module + 1 aggregate SBOM at the root everything generated from content in your POMs

joakime commented 1 year ago

yes, given your build is done with Maven, cyclonedx-maven-plugin looks like a good and easy solution you'll have one SBOM per module + 1 aggregate SBOM at the root everything generated from content in your POMs

The Jetty 12 environment means we'll have ...

This is because the transitive dependencies from the ee# level are different, and cannot converge.

jmcc0nn3ll commented 10 months ago

@waynebeaton recently put out a call to the Architecture Council to look at this issue. There is a lot of information regarding the direction Eclipse is taking here.

https://gitlab.eclipse.org/eclipsefdn/emo-team/sbom

olamy commented 10 months ago

I had a bit of try with that (see PR https://github.com/jetty/jetty.project/pull/10877). The plugin cannot make aggregate except from the top pom (or you do cd jetty-core && mvn ... and this all modules mentioned above. Maybe we can propose a patch to have the option to have aggregate sbom for sub modules. But thinking of it, I'm not sure if we really need those multi aggregate sbom for each eeX, core and integration as everything will be already in the top sbom. And this will be more easy to find something in it rather than having to navigate in multiple sbom. By the way when looking at the sbom for jetty-ee10-servlet you already have all infos for dependencies. the other very simple option (as recommended here https://gitlab.eclipse.org/eclipsefdn/emo-team/sbom/-/blob/main/docs/sbom.adoc) is single aggregated sbom from the pom (and no sbom for each module)

joakime commented 10 months ago

@olamy can the top level bom at least excluded things that are never deployed? (test modules)

jmcc0nn3ll commented 10 months ago

Based on yesterday's Architecture Council discussion, it only makes sense for the SBOM to be generated for the jetty-home artifact.

There is a slew of broad-scoped extensions to SBOMs, but no actual standard is emerging for intended use cases; most organizations appear just to be trying to 'tic a box' that they have SBOMs.

At this point, an SBOM that describes the contents of a zip file or a tarball is the bare minimum of what people are looking for.

This is subject to change, but if we achieve that, we meet the brief.

@waynebeaton fair?

jmcc0nn3ll commented 10 months ago

I should also mention that in maven land, the quality of the artifact metadata is the key factor in our producing a good SBOM, or an embedded jetty user building an SBOM for their product.

With that in mind, @waynebeaton opened an issue for maven enforcer rules to help projects make sure they have the basic needs met. Assuming the enforcer plugin can do it.

olamy commented 10 months ago

@olamy can the top level bom at least excluded things that are never deployed? (test modules)

yes. Have a look at the PR (https://github.com/jetty/jetty.project/pull/10877), every single module with <maven.deploy.skip>true</maven.deploy.skip> now have as well <cyclonedx.skip>true</cyclonedx.skip> which means the module is excluded from the aggregated sbom.

Log output

[INFO] --- cyclonedx:2.7.10:makeAggregateBom (cyclonedx-makeAggregateBom) @ jetty-project ---
[INFO] CycloneDX: Resolving Aggregated Dependencies
[INFO] Excluding jetty-http2-tests
[INFO] Excluding jetty-http3-tests
[INFO] Excluding jetty-tests
[INFO] Excluding jetty-test-client-transports
[INFO] Excluding jetty-test-jmx
[INFO] Excluding jetty-websocket-core-tests
[INFO] Excluding jetty-websocket-jetty-tests
[INFO] Excluding jetty-ee10-examples
....

I have made some changes to produce only a single sbom from top project. But I'm not quite sure about this approach as th sbom contains some artifacts references we do not distribute with our jetty-home.zip but those dependencies comes from optional jetty modules users can install on demand. Maybe we should generate the sbom only from jetty-home dependencies. WDYT?

olamy commented 10 months ago

@waynebeaton what do you think about the sbom content? Does it need to cover only what we distribute directly via the jetty-home distribution zip or do we need to cover everything, including optional modules that will be installed by users only on demand (but still deployed to Maven central though)

jmcc0nn3ll commented 10 months ago

The only thing we should worry about right now is an SBOM that is accurate for the jetty-home distribution, ideally including it within the artifact. The stock distribution for the time being. Until there is more clarity on what someone requires from an artifact's SBOM there is no point in producing them. The SBOM is generated from the maven metadata, so if they are trying to produce an SBOM for their product, they will just mine the maven metadata.

This will work right up until SBOMs contain industry-demanded data that is not tracked within maven. No one really knows what that is yet, way too much churn going on. Let's just do the bare minimum for the time being until someone can make a meaningful case for more. All these things are for most organizations at this point is a compliance checkbox.

joakime commented 10 months ago

@jmcc0nn3ll just to be clear I'm interpreting what you wrote as ...

Is that what you mean?

jmcc0nn3ll commented 10 months ago

We can put the SBOM in Maven Central as a classified artifact alongside the jetty-home, but it should go in it as well I think.

Currently, based on what we know about SBOMs from the last AC meeting, the SBOM is a file that describes what is included in the thing you just downloaded. So, in our case, it is a descriptor detailing what is on disk in that zip or tarball.

The current intended purpose is that the downloader knows precisely what is inside the thing they just downloaded and installed. I think of the SBOM as a way that the downstream consumer can cross that file against a vulnerability database and know if that zip or tarball is vulnerable.

The scope of this is likely going to grow broader as the whole SBOM ecosystem develops, but for now, it is enough to have an SBOM that describes what is in this thing you just downloaded. I see no value in SBOMing all the things when there isn't a clear intended purpose or use case for that.

olamy commented 10 months ago

This has been implemented in the PR #10877 .

olamy@pop-os:~/dev/sources/jetty/jetty.project$ unzip -l jetty-home/target/jetty-home-12.0.4-SNAPSHOT.zip  | grep sbom
   731336  2023-06-05 23:12   jetty-home-12.0.4-SNAPSHOT/jetty-home-sbom.json
   630796  2023-06-05 23:12   jetty-home-12.0.4-SNAPSHOT/jetty-home-sbom.xml

I don't like much the classifier used by the CycloneDX plugin for the attached sbom. The sbom in Maven central will have the following name jetty-home-12.0.4-SNAPSHOT-cyclonedx.xml This looks wrong to me as the classifier doesn't have any real meaning of what it is included in this file. I would prefer jetty-home-12.0.4-SNAPSHOT-sbom.xml The PR https://github.com/CycloneDX/cyclonedx-maven-plugin/pull/433 makes this configurable

olamy commented 10 months ago

well it looks I forgot about this sbom formats/specs "war" :) this classifier is definitely not a good idea. Just in case we need few other sbom format files ... I have even changed the file name included in jetty-home distro to include cyclonedx naming to be the same as the one deployed to Maven central.

olamy@pop-os:~/dev/sources/jetty/jetty.project$ unzip -l jetty-home/target/jetty-home-12.0.4-SNAPSHOT.zip  | grep cyclonedx
   731336  2023-06-05 23:12   jetty-home-12.0.4-SNAPSHOT/jetty-home-12.0.4-SNAPSHOT-cyclonedx.json
   630796  2023-06-05 23:12   jetty-home-12.0.4-SNAPSHOT/jetty-home-12.0.4-SNAPSHOT-cyclonedx.xml