sigstore / model-transparency

Supply chain security for ML
Apache License 2.0
99 stars 27 forks source link

Alignment with ML-BOM and M-BOM support in CycloneDX #56

Open stevespringett opened 8 months ago

stevespringett commented 8 months ago

Great to see Google interested in model transparency. Thank you for that.

I think there's two (possibly three) opportunities to align with OWASP CycloneDX.

Model card transparency

CycloneDX has extended its component model to incorporate model cards natively in the specification. Its model card support is derived from the Tensorflow Model Card Toolkit with sprinkles of VerifyML and a some additional fields not found in either. Model transparency is supported in CycloneDX today and there's a small but growing number of tools that support it.

See also:

Manufacturing transparency

CycloneDX has also incorporate support for Manufacturing Bill of Materials (MBOM) which can describe the precise steps (or formula) taken to create, test, train, evaluate, or deploy something. MBOM is an industry term which we've borrowed and brought into the software supply chain conversation. CycloneDX MBOM support can describe anything from how hardware was manufactured, software was created or deployed, how models were trained and evaluated, or how cloud services were orchestrated. CycloneDX MBOM support would provide a way to describe how a model is SLSA compliant.

See also:

Attestations (future v1.6+)

Many models are not publicly accessible (e.g. ChatGPT) and therefore it may be difficult to obtain model cards or manufacturing information. CycloneDX is evolving from a BOM standard into a transparency expression language. One such capability that v1.6 will support later this year is attestations. Think SSDF, PCI, or other industry, regulatory, or legally binding type of attestation, not in-toto - very different but complimentary types of attestations. Anyway, it would be possible to attest to SLSA or attest to having ethics and privacy incorporated into trained models using CycloneDX.


In general, I think there's a lot of opportunity for advancement and industry alignment if Google is interested in working with OWASP and Ecma.

mihaimaruseac commented 8 months ago

Thank you very much for opening the issue! Hope to chat about this during the ML-BOM seminar!