Support GnuPG signatures on anything that changes source content

[JanZerebecki]

Support optional end-to-end trust / content authentication in addition to transport authentication.

Support GnuPG (OpenPGP) signatures on anything that changes sources (build inputs). For some content adding multiple signatures by different people might be useful, for others like review request this can be mapped in multiple reviews. Use hash chains like in git to include history and other relevant parts in the content that is signed. (Signing other changes like comments would also be nice, but important things first.)

As a first step: On the server only check if the signature is correct but don't use it for authorisation. On the CLI when using content from the server add verification and checking of the signatures key against a set of GnuPG keyrings; add information related to the key in certain output. Have a way to configure the client to fail when a signature is missing e.g. on content from a user, on content in a project/package with wildcards; when content in a project/packages does not have n signatures from a specified list of keys; support calling out to supply this information shortly before verification.

As an alternatives I have considered 1) other standards supporting signatures (S/MIME and its relatives) 2) giving up on falsifiable software computations and security 3) increasing security in each link of the transport way / of the transitively trusted set.

[adrianschroeter]

It sounds that you could write a source service doing that. You can configure project wide services, which run on each source change. The package would turn into broken, if the service exits with an error.

Does this sound reasonable?

[JanZerebecki]

That wouldn't protect the history (leading to no rollback protection), nor the meta data in submit requests (the submit target, who accepted it, etc). But what would make it unusable in practice is that the signature would either need to be stored externally (which could solve protecting the history by storing in this external place the history with the signatures over it, but then that would conflict with one of the goals of obs, having the history of a project available in it and being able to build without external network) or accepting a submit request could not add a signature from the project maintainer.

Maybe using an external store is (despite the above) the way to go. For that to be sufficiently usable, creating a submit request, accepting a review, accepting a request, and creating a commit need to add a signature to this external store. Are there hooks so this functionality can be added on the client? Is there a hook on the server so that accepting a submit request can be refused when a correct signature wasn't added to the external store?

Can the content of a submit request be edited (to add a signature) by review acceptance or request acceptance? Does obs keep a hash chain of the revisions of a source package? Does obs keep a hash of each revision? Which hash functions are supported? Are multiple hash functions supported in parallel? Is there a way to upgrade in case a hash function gets broken?

[JanZerebecki]

There is git support being worked on on that will solve this completely. https://build.opensuse.org/project/show/devel:ALP currently uses a git repo for the project and git submodules for each package. What is missing is signature checking before any parsing or building.

[JanZerebecki]

Signed changes is only a small part of higher level requirements for end to end supply chain integrity, which are:

• every commit in package and code stream repositories (aka OBS projects) is either automatically reproducible for a rebuilder (see below) or is signed by a personal key
  • e.g. for merges done by bots this means verifying the signature on input commits and reproducing the merge
• required reviews are secured with personal keys
  • e.g. signed merge commit or a no-change commit with Signed-off-by line or signed pushes, but the later is less widely supported
• the target branch of changes is cryptographically protected
  • this includes never accepting a force push for a branch that gets used for a product
  • e.g. for the topmost personally signed commit being added to a branch mention the branch in a Target-branch trailer line or use signed pushes
• where verification of upstream source is possible allow describing how to do it in a machine readable way and verify it reproducibly
• the above are verified before being used to create an artefact
  • also provide a commit hook people can use to do this verification
• before a change is accepted into a product test it is reproducible by verifying the signature done by a rebuilder (that is a software that is easy to install and fully automated that listens for source changes in a product and automatically reproduces the artefacts and makes a signature to attest that and publishes those signatures for wide redistribution, see https://github.com/bmwiedemann/reproducibleopensuse)
  • artefacts here include also those besides binary packages, like disk images and containers
• any CI infrastructure that is asked for results in the work flow of a package needs to provide a retester (a rebuilder but for CI test results, though logs do not need to be reproducible)
• archive historical source and artefacts (see https://github.com/bmwiedemann/opensusearchive)
  • rebuilders need them for e.g. Tumbleweed
  • the archive should also provide access to current artefacts, so that rebuilders have a way to uniformly access the necessary input

Most of it probably will be provided by OBS (even if indirectly e.g. by https://github.com/openSUSE/obs-build/ ). But, I'm unsure where some of these requirements can be provided. Some part of acceptance into a product (AKA gating) is currently provided by bots for Factory in https://github.com/openSUSE/openSUSE-release-tools/ and not sure for Leap, there are bots for maintenance updates for SLE. Testing is mostly provided by openQA and some package specific steps not integrated with OBS.

(Of interest may also be some mentions of sigstore: https://github.com/openSUSE/open-build-service/search?q=sigstore https://github.com/openSUSE/obs-docu/search?q=sigstore )

[Conan-Kudo]

This is impossible to implement. This would require changing fundamentally how OBS works, since we'd need to preserve the build environments of every build like Koji does.

[JanZerebecki]

OBS publishes the build environment of every build in the _buildenv file. So, what do you mean?

[Conan-Kudo]

You will not be able to reconstruct the environment with the _buildenv file, the artifacts will not exist.

[JanZerebecki]

If everything is reproducible you only need the source which is already available in the OBS history and could reproduce the artifacts. But it is useful to have an artefact archive for allowing incremental improvement and allowing easier locating of errors. (This is important as one likely only enforces reproducibility in a statistical sense, for performance reasons.) That is why I listed it in the requirements, with a link to the archive Bernhard implemented and currently uses successfully:

• archive historical source and artefacts (see https://github.com/bmwiedemann/opensusearchive)

It is not necessary to do that in OBS, as OBS already provides the necessary APIs to be able to do this externally. Though you could be too slow and miss an artifact download. So it is better to implement this additional optional feature in OBS.

How does that require fundamentally changing OBS?

[JanZerebecki]

Example for a provenance file: https://sources.suse.com/_slsa/SUSE:SLE-15-SP4:Update/standard/x86_64/5a/5a9e94b5dd6396e0b2f60e808bf06c220137e19943adf13c7ccd1d35645ab2c9.prov

[adrianschroeter]

On Dienstag, 14. Juni 2022, 17:31:40 CEST Neal Gompa (ニール・ゴンパ) wrote:

You will not be able to reconstruct the environment with the _buildenv file, the artifacts will not exist.

They do exist in SLSA mode, but this works only for stable distributions. (Not Factory or devel projects)

--

Adrian Schroeter @.***> Build Infrastructure Project Manager

SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nürnberg, Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Ivo Totev