Reproducible (and signed?) builds

[effigies]

From both a reproducibility and security perspective - and I think we should take security seriously if we're building a package manager that will pull code off of the Internet and run it on health-related data - we should work towards reproducible builds. Python wheels and Docker/Singularity images should be as close to bit-for-bit identical as possible every time they're built, and a third party should be able to produce verifiably identical packages/images from the same git hash.

A few resources to look over in this topic:

• Reproducible Builds - a set of software development practices that create an independently-verifiable path from source to binary code
  • Diffoscope
• signalapp/Signal-Android/wiki/Reproducible-Builds

This should apply equally to nfm itself and any packages produced by it.

We should also consider signed tags/packages. Tags would be relatively easy to enforce, triggering a Travis/CircleCI failure if there is no signature or it doesn't verify. Signed packages would directly contravene #8 (a cursory glance through PyPI docs does not indicate a way for us to grab the packages, verify and subsequently upload signatures). It's possible that a signed tag and reproducible package would provide sufficient assurance, but it would be multi-step, rather than built-in.

We could also consider a separate system from Travis for uploading to Python, which would verify the tag, build and sign the package with a key clearly marked as belonging to an automatic builder and upload to PyPI.

[kaczmarj]

@effigies - I think signing nfm and workflows it distributes is a great idea. Can you explain what you want to do when you wrote

grab the packages, verify and subsequently upload signatures

from PyPi? The twine documentation suggests that packages can be signed before being uploaded to PyPi. According to https://github.com/pypa/twine/issues/157#issuecomment-170519789, the signature for the wheel is available at {WHEEL_URL}.asc.

[effigies]

I was imagining the scenario where Travis could build the packages and upload them to PyPI, and somebody could, after the fact, download the packages from PyPI, rebuild from source and verify that the packages on PyPI match, and then sign the packages and upload just the signatures. This would preserve us from uploading a package signing key to Travis or making the whole release process manual.