mar autograph hash signing

mozilla-releng / signingscript

Signing script to run in scriptworker.

1 stars 11 forks source link

mar autograph hash signing #69

Closed g-k closed 6 years ago

g-k commented 6 years ago

refs: https://bugzilla.mozilla.org/show_bug.cgi?id=1471196 (load testing and perf improvements for MAR signing)

From the go client signing mar hashes for partial1.mar (~6-7MB) offers a ~10x improvement (2s vs 20s) over signing files against the stage HSM (edit: this was more like 8s of improvement for Ben from signingscript dev on us-east-1). It also allows lets us sign larger MAR files without having to transfer them over the network.

r? @escapewindow

NB: we might want to land the verification branch first

edit:

other changes:

bump autograph image version for integration tests
factor out flag to raise on empty list for get_suitable_signing_servers

coveralls commented 6 years ago

Coverage remained the same at 100.0% when pulling 40d5b4993eeb26cda9715020499b65a58ea91ba5 on g-k:mar-autograph-hash-signing into 4ee5fd9654b85fca63a00202ea6014f01f231984 on mozilla-releng:master.

escapewindow commented 6 years ago

Sorry, I missed that this was not wip and is ready for r? =\ Sorry for the delay; on my radar.

g-k commented 6 years ago

No worries, it's good to have more data from signing whole files to compare against.

g-k commented 6 years ago

@escapewindow ready for another look. Think I got everything from review.

Kept the _hash_only part of the format, but we can update that key in FORMAT_TO_SIGNING_FUNCTION to autograph_mar384 if we want to make hash signing the mar default.

escapewindow commented 6 years ago

The interdiff looks good to me. I'm going to try testing this against autograph-stage.

escapewindow commented 6 years ago

Testing is currently held up by the lack of lzma support in python:

I was able to install this PR + its new upstream dependencies (specifically backports.lzma) by installing xz-devel
it appears our version of python 3.6.5 is missing lzma support because it was compiled on a machine without xz-devel.

Currently looking at recompiling python 3.6.5 with lzma support on Docker; I may also try the rpmbuilder ec2 instance.

g-k commented 6 years ago

Bleh, yeah I had to install the liblzma-dev pkg for my local env on xubuntu.

escapewindow commented 6 years ago

https://github.com/mozilla-releng/build-puppet/pull/236 will roll out the lzma-support python 3.6.5. Once that lands, we can merge this PR and release it. (The puppet patch will need to a) install xz-devel on signing scriptworkers, b) update the passwords files with the new format, and c) bump the signingscript version.)

At that point, we'll be able to choose if/when to switch over to signing hashes, either by an in-tree patch, or by changing the behavior of autograph_mar384 to sign hashes if we want a more aggressive rollout.