search

google / oss-rebuild

Securing open-source package ecosystems by originating, validating, and augmenting build attestations.
Apache License 2.0
21 stars 3 forks source link

Make normalization logic more accessible #39

Open hboutemy opened 2 months ago

hboutemy commented 2 months ago

current attestations are done just after rebuild, on zip files that are de-facto not reproducible because they contain build timestamp

normalizing timestamp with ziptime for example (or any equivalent tool) would ease rebuilding locally and confirming results

rebuild instructions could even download reference archive from registries and apply the normalization tool then check that normalized rebuild output is exactly the same as normalized reference archive

msuozzo commented 2 months ago

Hello!

To briefly summarize the current two-staged rebuild process:

  1. Faithful build of the upstream package (note: this is encapsulated in the Dockerfile we provide)
  2. Normalization process that takes in both upstream and rebuild and applies transforms to canonicalize the artifacts

The faithful build is intended to match how the upstream builds and, as such, will not have any normalizations (e.g. zip build time set, anything in strip-nondeterminism) applied. One reason for this separation is to run the build in an untrusted sandbox while the (security-critical) normalization can happen in trusted code.

To try to get at the root of the issue: I think you're (reasonably) asking "Why isn't the provided build deterministic/reproducible?"

To attempt an answer: The two stages of the rebuild process together are deterministic but we don't currently provide an easy way to run the second normalization step.

To look forward to a solution: I think users should be able to easily run both stage 1 alone and stage 1+2 so we should take an FR to expose that.

Let me know if that answers your question and thanks for filing the issue!

hboutemy commented 2 months ago

yes, this perfectly answers the question, thank you and waiting for the feature, this does the minimum documentation that was necessary :)