search

consumer-reports-innovation-lab / TheDigitalStandard

The Digital Standard is an ambitious, community-led effort to build a framework to test and rate products and services on the basis of privacy, security, and data practices.
Creative Commons Attribution 4.0 International
128 stars 46 forks source link

Best Build Practices: Reproducible Builds #111

Closed azadi closed 4 years ago

azadi commented 5 years ago

Hi.

To the "Best Build Practices" section, perhaps we should add a section on reproducible builds. Reproducible builds ensure deterministic output where a given source (code) always produces the same output (binary). So that means if you build a software on two different machines, the resulting binary should be byte-for-byte identical. This helps validate that the binary corresponds to the source code and that for example no backdoor was introduced in the compilation process.

More information is available at https://reproducible-builds.org/.

I am not sure how proposing an addition works but I thought I should first open this issue to discuss and based on the feedback, I am happy to submit a PR.

KatieMcInnis commented 5 years ago

Thanks, @azadi ! Do you recommend adding "uses reproducible builds" as an indicator? Or should it be under Procedure Overview? Perhaps it goes under the indicator "The product was built with effectively implemented safety features." as a procedure?

azadi commented 5 years ago

Thanks, @azadi ! Do you recommend adding "uses reproducible builds" as an indicator? Or should it be under Procedure Overview? Perhaps it goes under the indicator "The product was built with effectively implemented safety features." as a procedure?

I think it should be a Procedure Overview with the text: "Was the software built reproducibly?" under the "The product was built with effectively implemented safety features" indicator but I am curious to hear what others think.

Katie, if I may bring up a related point, I now notice that there are no references to technical terms on this page — do you think it will be helpful to add those? For example, it may be helpful to the reader to find out what reproducible software actually is and we should add a link to https://reproducible-builds.org/. Similarly for other terms on the page, like ASLR or HEASLR. I realize this brings up the question of what resource to link to and that may require further discussion.

KatieMcInnis commented 5 years ago

@azadi

  1. Wonderful! Sounds like you have a pull request ready to go : )
  2. What a great point! @secretrobotron and I were just discussing the need for a glossary in Issue 103. Do you think a glossary would account for this concern? Or should we explore using more hyperlinks in the Standard?
secretrobotron commented 5 years ago

Thanks a ton, @azadi. Correct me if I'm wrong, but this would be most relevant if you actually have the source code, right? (e.g. an open source project.) Do you know if people are doing this in other contexts as well? In a private context, I can see a situation where you'd hire several third parties to compile your software for you and then compare the hash from each to make sure they're identical.

secretrobotron commented 5 years ago

Ok, I took a crack at it. See https://github.com/TheDigitalStandard/TheDigitalStandard/pull/115.

KatieMcInnis commented 5 years ago

Thanks, @secretrobotron.