Create utility for generating SBOM from artifacts

sethmlarson commented 8 months ago

Part of https://github.com/python/cpython/issues/112302

So this is a decent amount of code to review, but it does indeed check all the boxes that I want SBOMs for source tarball artifacts to check. This likely won't be the final state of this module, I think the end-goal is to stitch it in to the release process in various points.

You can see the generated SBOM in this Gist: https://gist.github.com/sethmlarson/103891c6cac4d41b11daab89e6c84868

NOTE: that I had to modify the 3.13.0a2 tarball by adding a Misc/sbom.spdx.json file into the archive. 3.13.0a3 will be the first release which contains the base SBOM file in the tarball.

Here are the criteria the generated SBOM meets:

Meets NTIA minimum elements
Matches SPDX 2.3 JSON schema
Grype is able to use the SBOM and find vulnerabilities
Scores 9.6 out of 10 on SBOM Quality, and the only metric is one we can't improve on our own (multiple IDs per package)

$ grype sbom:./Python-3.13.0a2.tgz.spdx.json 
 ✔ Vulnerability DB                [no update available]  
 ✔ Scanned for vulnerabilities     [3 vulnerability matches]  
   ├── by severity: 0 critical, 1 high, 1 medium, 1 low, 0 negligible
   └── by status:   1 fixed, 2 not-fixed, 0 ignored 
[0000]  WARN some package(s) are missing CPEs. This may result in missing vulnerabilities. You may autogenerate these using: --add-cpes-if-none
NAME  INSTALLED  FIXED-IN  TYPE    VULNERABILITY        SEVERITY 
pip   23.2.1               python  CVE-2018-20225       High      
pip   23.2.1     23.3      python  GHSA-mq26-g339-26xf  Medium    
pip   23.2.1               python  CVE-2023-5752        Low

$ sbomqs score Python-3.13.0a2.tgz.spdx.json 

SBOM Quality Score:9.6  components:7    Python-3.13.0a2.tgz.spdx.json 
+-----------------------+--------------------------------+-----------+--------------------------------+
|       CATEGORY        |            FEATURE             |   SCORE   |              DESC              |
+-----------------------+--------------------------------+-----------+--------------------------------+
| NTIA-minimum-elements | comp_with_name                 | 10.0/10.0 | 7/7 have names                 |
+                       +--------------------------------+-----------+--------------------------------+
|                       | comp_with_supplier             | 10.0/10.0 | 7/7 have supplier names        |
+                       +--------------------------------+-----------+--------------------------------+
|                       | comp_with_uniq_ids             | 10.0/10.0 | 7/7 have unique ID's           |
+                       +--------------------------------+-----------+--------------------------------+
|                       | comp_with_version              | 10.0/10.0 | 7/7 have versions              |
+                       +--------------------------------+-----------+--------------------------------+
|                       | sbom_authors                   | 10.0/10.0 | doc has 2 authors              |
+                       +--------------------------------+-----------+--------------------------------+
|                       | sbom_creation_timestamp        | 10.0/10.0 | doc has creation timestamp     |
|                       |                                |           | 2024-01-10T22:08:42Z           |
+                       +--------------------------------+-----------+--------------------------------+
|                       | sbom_dependencies              | 10.0/10.0 | doc has 4422 relationships     |
+-----------------------+--------------------------------+-----------+--------------------------------+
| Quality               | comp_valid_licenses            | 10.0/10.0 | 7/7 components with valid      |
|                       |                                |           | license                        |
+                       +--------------------------------+-----------+--------------------------------+
|                       | comp_with_any_vuln_lookup_id   | 10.0/10.0 | 7/7 components have any lookup |
|                       |                                |           | id                             |
+                       +--------------------------------+-----------+--------------------------------+
|                       | comp_with_deprecated_licenses  | 10.0/10.0 | 0/7 components have deprecated |
|                       |                                |           | licenses                       |
+                       +--------------------------------+-----------+--------------------------------+
|                       | comp_with_multi_vuln_lookup_id | 1.4/10.0  | 1/7 components have multiple   |
|                       |                                |           | lookup id                      |
+                       +--------------------------------+-----------+--------------------------------+
|                       | comp_with_primary_purpose      | 10.0/10.0 | 7/7 components have primary    |
|                       |                                |           | purpose specified              |
+                       +--------------------------------+-----------+--------------------------------+
|                       | comp_with_restrictive_licenses | 10.0/10.0 | 0/7 components have restricted |
|                       |                                |           | licenses                       |
+                       +--------------------------------+-----------+--------------------------------+
|                       | sbom_with_creator_and_version  | 10.0/10.0 | 1/1 tools have creator and     |
|                       |                                |           | version                        |
+                       +--------------------------------+-----------+--------------------------------+
|                       | sbom_with_primary_component    | 10.0/10.0 | primary component found        |
+-----------------------+--------------------------------+-----------+--------------------------------+
| Semantic              | comp_with_checksums            | 10.0/10.0 | 7/7 have checksums             |
+                       +--------------------------------+-----------+--------------------------------+
|                       | comp_with_licenses             | 10.0/10.0 | 7/7 have licenses              |
+                       +--------------------------------+-----------+--------------------------------+
|                       | sbom_required_fields           | 10.0/10.0 | Doc Fields:true Pkg            |
|                       |                                |           | Fields:true                    |
+-----------------------+--------------------------------+-----------+--------------------------------+
| Sharing               | sbom_sharable                  | 10.0/10.0 | doc has a sharable license     |
|                       |                                |           | free 1 :: of 1                 |
+-----------------------+--------------------------------+-----------+--------------------------------+
| Structural            | sbom_parsable                  | 10.0/10.0 | provided sbom is parsable      |
+                       +--------------------------------+-----------+--------------------------------+
|                       | sbom_spec                      | 10.0/10.0 | provided sbom is in a          |
|                       |                                |           | supported sbom format of       |
|                       |                                |           | spdx,cyclonedx                 |
+                       +--------------------------------+-----------+--------------------------------+
|                       | sbom_spec_file_format          | 10.0/10.0 | provided sbom should be in     |
|                       |                                |           | supported file format for      |
|                       |                                |           | spec: json and version:        |
|                       |                                |           | json,yaml,rdf,tag-value        |
+                       +--------------------------------+-----------+--------------------------------+
|                       | sbom_spec_version              | 10.0/10.0 | provided sbom should be in     |
|                       |                                |           | supported spec version for     |
|                       |                                |           | spec:SPDX-2.3 and versions:    |
|                       |                                |           | SPDX-2.1,SPDX-2.2,SPDX-2.3     |
+-----------------------+--------------------------------+-----------+--------------------------------+

hugovk commented 8 months ago

How should this be run?

❯ p sbom.py
Traceback (most recent call last):
  File "/Users/hugo/github/release-tools/sbom.py", line 274, in <module>
    tarball_path = sys.argv[1]
                   ~~~~~~~~^^^
IndexError: list index out of range

❯ p sbom.py -h
Traceback (most recent call last):
  File "/Users/hugo/github/release-tools/sbom.py", line 277, in <module>
    create_sbom_for_source_tarball(tarball_path), indent=2, sort_keys=True
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/hugo/github/release-tools/sbom.py", line 103, in create_sbom_for_source_tarball
    raise ValueError(f"Unknown tarball format: '{tarball_name}'")
ValueError: Unknown tarball format: '-h'

❯ p sbom.py /tmp/downloads/Python-3.13.0a2.tgz
Traceback (most recent call last):
  File "/Users/hugo/github/release-tools/sbom.py", line 277, in <module>
    create_sbom_for_source_tarball(tarball_path), indent=2, sort_keys=True
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/hugo/github/release-tools/sbom.py", line 117, in create_sbom_for_source_tarball
    sbom_bytes = tarball.extractfile(tarball.getmember("Misc/sbom.spdx.json")).read()
                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/tarfile.py", line 1984, in getmember
    raise KeyError("filename %r not found" % name)
KeyError: "filename 'Misc/sbom.spdx.json' not found"

I got a similar KeyError running from the CPython repo.

sethmlarson commented 8 months ago

@hugovk Sorry for sending you on a wild goose chase! I actually had to modify the 3.13.0a2 release tarball by adding the Misc/sbom.spdx.json file into the archive in order to test the tool. 3.13.0a3 will be the first actual CPython release that has the source tree dependencies SBOM. I'll note that in the top issue to not confuse other folks.

sethmlarson commented 8 months ago

Okay, this is ready for review. Depends on https://github.com/python/release-tools/pull/84 being merged first. cc @hugovk Note that the only tarball with an SBOM available is 3.13.0a3

You can test the SBOM generation with $ python sbom.py Python-3.13.0a3.tgz

sethmlarson commented 8 months ago

Thanks for the reviews! I've applied the suggestions :)

python / release-tools

Create utility for generating SBOM from artifacts #82