search

spdx / ntia-conformance-checker

Check SPDX SBOM for NTIA minimum elements
Apache License 2.0
53 stars 20 forks source link

Update to new edition of minimum elements (2024) #214

Open bact opened 2 weeks ago

bact commented 2 weeks ago
jspeed-meyers commented 2 weeks ago

Thank you for pointing this out, @bact! I defer to @goneall and @kestewart. Do you want this tool to reflect the revised criteria in the new document cited above?

If so, that could be a good project for a Google Summer of Code intern.

bact commented 2 weeks ago

Yes, I think that will be useful and help adoption of SPDX. Although I'm not certain about the scope of SPDX versions we would like to support.

Currently, ntia-conformance-checker supports SPDX 2.3. To have the support for the Third edition of minimum elements in SPDX 2.3 is one thing. To have the support for the Third edition of minimum elements also in SPDX 3.0 is another thing.

Considering the experimental support of SPDX 3.0 in spdx-tools dependency, an SPDX 2.3 is probably more feasible in the near term. Although the additional SPDX 3.0 support will do more favour for 3.0 adoption.

--

There is also a similar document from the German Federal Office for Information Security (BSI) that we could consider together:

(and the Summer of Code is interesting)

jspeed-meyers commented 2 weeks ago

To be clear, I support updating the ntia-conformance-checker to implement the new edition of the minimum elements:

I can't currently justify B personally (at least as a volunteer maintainer), though I would be glad to supervise a GSOC intern or outside contributor to do it, reviewing PRs as necessary and providing advice.

bact commented 2 weeks ago

Thank you. Having played around a bit with tools-python SPDX model, I'm happy to look into this.

Let's see how Gary and Kate think about this.

goneall commented 2 weeks ago

I personally support updating the minimum elements. For compatibility, I would suggest that we have a command line option to select which version of the minimum elements is used - the original NTIA or the 3rd edition of the framing document.

jspeed-meyers commented 2 weeks ago

@bact or others: I'm glad to advise on how to implement this change. I suggest creating a design document first (perhaps a comment-only Google document or something similar) since there is potentially a lot to unpack and then arriving at a rough consensus before sending any PRs in.

bact commented 2 weeks ago

Thanks. I will try to come up with the design doc, maybe next week.

jspeed-meyers commented 2 weeks ago

Sounds good, @bact. And LMK if you want to brainstorm or if I can help in any way.

bact commented 1 week ago

@jspeed-meyers I have put some notes here: https://docs.google.com/document/d/1pueRxlxoM9n1eG9g6AihjLvybEBTd77m22mRYBQltpg/edit?usp=sharing

Note that sbomqs already supports FSCT v3 on SPDX 2.x, so may need to see what is still missing.

jspeed-meyers commented 1 week ago

@bact: Nice document.

IIUC, you are proposing to expand this tool's mandate from checking the conformance of an SPDX SBOM with the NTIA minimum elements to checking SPDX SBOM conformance with a range of frameworks. Do I understand your intention correctly?

In general, I want this tool be broadly useful. So I support this motion in the abstract.

I do have two concerns. First, would any users besides yourself find this useful? I worry about expanding the mandate of the tool without clear and strong evidence that MANY users would find this helpful. Second, who will do the creation and maintenance of such a tool? I have become a co-maintainer of this tool (and the company where I work USED to use this tool internally), but there are no longer clear incentives for me to do anything other than basic maintenance. I worry about adding lots of functionality, which will inevitably have bugs, and there being no set of maintainers able to debug and propose fixes.

Anyways, nice document! I support the idea in general.

For me, the most important opinions are those of @goneall and @kestewart. I defer to them.

bact commented 1 week ago

Thank you. Very useful comments.

1) On the demand side

Frankly I don't know. I believe there will be an increasing demand due to regulations and business needs, counting on the existence of a tool like sbomqs.

But of course, as currently sbomqs support more standards and formats (SPDX 2 and CycloneDX), there's no good reason to switch to the tool in this repo.

Unless there's a featue that other tools don't have yet (and that feature is essential enough, which I don't know what it is).

Personally, I want to use this tool to check the conformance of an SBOM against requirements in EU AI Act. This will link to the supply side, the creation and maintenance in the next point.

2) On the supply side

I am willing to create some of these features.

My idea is to start somewhere with less moving parts, to understand how things work, so standard documents like NTIA and FSCTv3 came to my mind. They are quite established and I don't have to worry much about interpretation.

After I understand how to technically check SBOM conformance, I will then continue to apply it with EU AI Act requirements.

I have an incentive for the creation because it will help with my study at university. But of course, one can question the maintenance in the long run after I left university (which personally I hope happens soon).

Maybe to ease the concern, each addition of standard support should be developed in a way that detached from the main program. A feature should be removed easily when required (the feature is no longer maintained and has bug/dependencies that will affect the main program, or the standard document is deprecated/revoked).

jspeed-meyers commented 1 week ago

But of course, as currently sbomqs support more standards and formats (SPDX 2 and CycloneDX), there's no good reason to switch to the tool in this repo.

Yeah, I think sbomqs is a perfectly good tool. The advantage, IMO, of this tool, ntia-conformance-checker, is not technical but organizational: it is housed within the spdx GitHub organization as an officially-supported implementation of NTIA conformance checking for SPDX SBOM documents. But if a user prefers sbomqs, then go for it!

Maybe to ease the concern, each addition of standard support should be developed in a way that detached from the main program.

I do think this would ease my concerns.

And, again, I'm glad to review PRs.

I still have the broader question though of: should this tool include SBOM quality standards beyond the NTIA minimum elements? I'm open to it. But I really do see this as a Gary and Kate question.

Also, I too originally got interested in this tool because of research (see here), so I understand your situation :)

goneall commented 6 days ago

I have a couple of opinions on this topic:

jspeed-meyers commented 6 days ago

@bact: Given @goneall's support, I would suggest, at least in the short term, a PR that focuses on the 3rd edition framing document conformance checking at the minimum level, implemented via a new option like the design document above. If you aren't totally opposed to more design document work, I would suggest sketching out what new functions need to be added to implement the 3rd edition framing document conformance checking. Once you and I have a rough consensus there, you can send in a PR.

And everything else can wait until a later day.

Does that satisfy your immediate research needs?

bact commented 6 days ago

@goneall @jspeed-meyers Thank you. Yes, I think it it sound to start from the "Minimum Expected" level of the FSCT v3.

Will sketching what needs to be added first as John suggested.