Create signature generation automation for server-side deployment

[oyiptong]

The deployment plan is to have an nginx server running lua serve the pages.

Some deployment automation will generate the html page signatures, and provide them to nginx via a custom S3 header, x-aws-content-signature.

Nginx will then serve the html pages, with the page signatures as the Content-Signature header, as detailed in #150.

The algorithm used for signing is ECDSA. This library does what we want: https://github.com/warner/python-ecdsa

An example server can be found at: https://bugzilla.mozilla.org/show_bug.cgi?id=1226928

[oyiptong]

Since we don't want developers to have a the private key on their machines, this is about writing the server-side code that will:

1. receive a request to deploy code
2. generate the signature for each index.html page (or pages that need signatures)
3. upload the pages to S3, with the x-aws-content-signature custom header

[oyiptong]

@mostlygeek: let's use this page to talk about this

@franziskuskiefer is working on content-signature verification and key storage on the Firefox side of things. He said he might want to help on the server-side as well.

[mostlygeek]

The last time we talked it was about S3 not being able to serve the header (Content-Signature) since we can't specify arbitrary HTTP response headers. With S3 since all custom metadata is prefixed with x-amz-meta-, we discussed using nginx to look for that header and rewrite it with lua.

nginx would essentially be a smart(ish) proxy between S3 and the CDN. The request path would be:

firefox => CDN => nginx => S3

Is that what we are talking about with this issue?

[oyiptong]

Yes that is correct.

In addition to the proxy, we need a small-ish server that upon receiving a deployment request, will create signatures. We don't want the private keys to be on developers' laptops.

[mostlygeek]

Can we make two separate issues to deal with them? One for serving the content (this one) and one for signing+publishing the content.

[oyiptong]

This is for signing + publishing, #150 is for serving.

[franziskuskiefer]

I think I'm a bit uneasy with using a random python library for signing. So I'd suggest using some simple openssl commands instead (I'm rewriting my test server to use the same logic as well). We can do something like:

openssl ecparam -name secp384r1 -genkey -out sk.pem
openssl dgst -hex -sha384 -sign sk.pem RemoteNewTab.html

to generate a key and the signature (note that the html file must contain the Content-Encryption\00 prefix). The resulting signature has to be converted to urlsafe base64 and we can get the verification key from sk.pem with python any other command line tool (openssl is here not really helpful).

Note that outdated openssl versions might not be able to do sha384.

[mostlygeek]

We use centos7 in production. Though if you guys ship it in a docker container you can use whichever version of OpenSSL is appropriate.

[franziskuskiefer]

I just updated the server in bugzilla, which can be used as follows:

• create a prefix file to prepend to every html file before signing with content "Content-Encryption:\x00"
• prepend the file to every file you sign (RemoteNewTab.html here)
  • paste prefix RemoteNewTab.html > RemoteNewTab.html.tosign
• do crypto magic with openssl
  • openssl ecparam -name secp384r1 -genkey -out sk.pem
  • openssl dgst -sha384 -sign sk.pem RemoteNewTab.html.tosign > signature.der
• run the server ./server.py 8000 RemoteNewTab.html signature.der

@oyiptong Can you work with this? You should be able to use the commands and example server to create a signing service. The python server has a line to print the public key, that's the one we need in Firefox for verification. That would need some kind of automation