Intent to implement: New Fast Fetch signature scheme

[taymonbeal]

This is a proposal to change the name and format of the HTTP response header containing the signature of an AMP creative requested through Fast Fetch. The purpose of this change is to enable improvements to the cryptographic verification process in the Fast Fetch extension.

Rationale

There are now multiple Fast Fetch signing services, and each service can maintain multiple keypairs. However, code inspecting a signature in the current format cannot immediately determine which signing service generated the signature, much less which keypair. The current client works around this by asynchronously attempting verification against each known key, and then repeating the entire process with freshly downloaded keysets if it fails. This is a major source of unnecessary implementation complexity in Fast Fetch, and has been responsible for at least one production outage. It also makes it infeasible in many cases to distinguish between different classes of errors (for instance, a signature not matching the creative vs. one that used a key not available in the cache), and results in suboptimal behavior (such as repeated keyset downloads) on failure. Under the new scheme, the client-side code can be much simpler, errors will be unambiguous and easily reportable, and keysets will be redownloaded only in the one error case where doing so might actually help (an unrecognized key from a known signing service).

New Requirements for Signing Services and Ad Networks

Each signing service will be required to assign an ID to each of its signing keypairs. A keypair ID must be a string of of one or more characters, each of which must be an ASCII uppercase or lowercase letter, an ASCII digit, -, ., or _. If two different public keys used by the same signing service were ever served from that signing service's public key endpoint with overlapping freshness lifetimes, they must have different keypair IDs. (For this purpose, a staging-only signing service such as google-dev is considered a different signing service from its production counterpart.) Each signing service may use any scheme it wishes to assign IDs to keypairs, as long as these requirements are followed.

Each JSON Web Key served from a signing service's public key endpoint must include its keypair ID as the value of that key's "kid" parameter.

Instead of sending an X-Ampadsignature header, ad servers responding to Fast Fetch requests with validated AMP creatives will be required to send an HTTP response header named AMP-Fast-Fetch-Signature. The value of this header must be the name of the signing service that validated the creative (as listed in the signing service registry), followed by :, followed by the keypair ID of the key used to sign the creative, followed by :, followed by the RSASSA-PKCS1-v1_5 signature encoded in base64 (not base64url) format. For example, a header sent by an ad server using Google's signing service might look like this:

AMP-Fast-Fetch-Signature: google:20170216:SK4x6fsU4L+OEu+TC8DbGI9zdlHv41Ta5tzS0I4QDOYO3W5V/T6y7SOeBIN8milVaR7hXTC/M9qbAnQ5Q/rPtahnTd/0mj5B5wLRjI8GKCRR3RFTLoVCMO31cYZTR5Ytay/IxJx5IWN3L76KVMy/AXs6K237p+EkxgUJEJBs4YjtSfEYEhAhEpn/nqVRqUJ//Hk9MA9p2yuTd9/+zMu/kvUpKBzu2xFOELvTDpmOKGDT33CarRm/iMifZE+v5DfagXeFkJtFOLDM9LSEXoJ4GPlQ7eLdAlwPlkYHjzjSEIzIZeE+2bpDkoTo5/iCrZsHhg8I2MLWa8JhTJFBjxb2Wg==

(The keypair ID 20170216 is speculative, provided for illustrative purposes only, and should not be taken to imply anything about what scheme Google's signing service will actually use to assign IDs to keypairs.)

New Client Behavior

When an <amp-ad> element is upgraded to Fast Fetch (during the AmpA4A constructor; this may occur during prerendering of the document), if Web Cryptography is available, an AJAX request is made to the public key endpoint of each signing service used by the applicable ad network, if this hasn't already been done for that signing service by a different ad slot on the page. It is intended that many such requests will be responded to by the browser's HTTP cache and will not require a network round-trip. The keysets are then imported. (This is not a change from existing behavior.) If a non-200 or malformed response is returned, an error will be logged against the signing service. (The notion of logging an error "against" a third party is not especially meaningful right now, but the plan is to eventually support error reporting to multiple parties via <amp-analytics>.)

When the ad response is available (after the keyset requests have been made, the document has become visible, and the ad slot has been positioned on the page), the following steps are taken:

1. If Web Cryptography is not available or the response does not include a AMP-Fast-Fetch-Signature header, render the creative in a cross-origin iframe after a runtime-imposed delay. Otherwise, continue.
2. If the AMP-Fast-Fetch-Signature header is malformed, contains the name of a nonexistent signing service, or contains the name of a signing service not used by the ad network, log an error against the ad network and render the creative in a cross-origin iframe after a runtime-imposed delay. Otherwise, continue.
3. Wait for the named signing service's keyset to be downloaded and imported, if it hasn't been already. If a network or other error occurred during that process, render the creative in a cross-origin iframe after a runtime-imposed delay. Otherwise, continue.
4. If the keypair ID is not in the keyset, make another AJAX request to the named signing service's public key endpoint, this time with the keypair ID in the query string to bust cache. Import each key whose keypair ID was not in the existing keyset. If a network error occurs, render the creative in a cross-origin iframe after a runtime-imposed delay. If a non-200 or malformed response is returned, log an error against the signing service and render the creative in a cross-origin iframe after a runtime-imposed delay. If the keypair ID from the AMP-Fast-Fetch-Signature header is not in the new keyset, log an error against the ad network and render the creative in a cross-origin iframe after a runtime-imposed delay. Otherwise, continue.
5. Cryptographically verify the creative and signature against the public key with the named keypair ID. If this verification fails, log an error against the ad network and render the creative in a cross-origin iframe after a runtime-imposed delay. Otherwise, render the creative in a same-origin iframe immediately.

Transitioning

There are two options for transitioning from the old scheme to the new one. We could add an additional code path to the Fast Fetch extension alongside the existing one (with logic to divert between the two based on which headers are present), and then let each Fast Fetch ad server switch from the old header to the new one. Alternatively, Fast Fetch ad servers could start sending the new header alongside the old one, and then, after they had all started doing so, we could replace the old client-side code path with the new one all at once.

We propose that the second option be taken, because it means we don't have to maintain two separate and very different code paths, along with additional logic to dispatch between them, in an already overly-complex and bug-prone part of the client. The burden on ad servers is expected to be minimal, since all they have to do is send an additional header very similar to the one they're already sending.

As such, the following steps need to be taken in order:

1. The AMP maintainers accept this I2I.
2. All signing services (Google and Cloudflare) start including kid in their public JSON Web Keys as specified above.
3. All Fast Fetch ad servers (currently all operated by either Google or Cloudflare) start sending the AMP-Fast-Fetch-Signature header as specified above.
4. Wait for any rollback horizons applicable to any ad servers to pass.
5. The AMP maintainers merge the A4A team's pull request containing the client-side changes (which will be written, opened, and reviewed in parallel with steps 2 through 4).
6. Wait for the next AMP runtime release to go through Dev Channel, then be released to production, then for its rollback horizon to pass.
7. Ad servers stop sending the X-Ampadsignature header at their convenience.

[taymonbeal]

CC @ampproject/a4a @dknecht @oliy

[dknecht]

CC: @ampproject/cloudflare

[jasti]

@lannka can you accept this ITI? Thanks.

[oliy]

This sounds pretty straightforward and makes sense against the current setup. When this is finalized, we'll start generating the new Header. Is there any consideration for the endpoints to support delivering multiple creatives/signatures?

[taymonbeal]

Right now, that's not supported, so it doesn't affect this I2I. When Single Request Architecture becomes available, I suspect it'll use the same signature format, with each creative's signature somehow being attached to that creative (perhaps as another field of a JSON object).

[lannka]

All looks good. One thing: can we be more restrictive about the "kid"'s charset? For sake of security, since you plan to put the "kid" in URL, and also for more readable error reporting.

[lannka]

If the keypair ID is not in the keyset, make another AJAX request to the named signing service's public key endpoint, this time with the keypair ID in the query string to bust cache.

This way, a new page view would still hit the old cache first, and then the new cache, right?

[taymonbeal]

@lannka: Sure, I've edited the proposal to tighten up the rules. Keypair IDs are now required to consist of one or more ASCII characters in the range [A-Za-z0-9._-]. CC @ampproject/a4a, @ampproject/cloudflare, and @vitaliybl.

Regarding the cache, yes, that would happen until the old cache expires.

[lannka]

Given the cache issue, @taymonbeal instead of requesting a set of keypairs, what do you think if we only request one keypair that is used in the current creative?

It's true that requesting the full set can happen in parallel with the ad request, but once there's key update (not sure how frequent it is), the cache becomes helpless.

Requesting one keypair after the creative response can potentially further simplify the client side logic. And most likely, it will always hit the cache hence no additional round-trip will be added.

[taymonbeal]

@keithwrightbos and @vitaliybl, do either of you have thoughts on this?

[keithwrightbos]

I think we should address this as an experiment once Fast Fetch launches to determine what impact delaying the key fetch has on overall render time.

[taymonbeal]

@oliy, Google is now sending the new header. What's Cloudflare's status on this?

[oliy]

We've been sending the new and old headers already. Is there a good way to validate that our new headers are working (over the old headers), short of dropping the old headers?

[ampprojectbot]

This issue hasn't been updated in awhile. @taymonbeal Do you have any updates?