Upgrade Tachyon Dynamic Image Resizing Infrastructure

[kingkool68]

At the end of November, 2017, we put infrastructure in place to be able to run Tachyon for dynamic image resizing via query strings (https://github.com/spiritedmedia/spiritedmedia/issues/2016). Unfortunately this caused other issues like animated gifs not to load (https://github.com/spiritedmedia/spiritedmedia/issues/2422), unsupported file types weren't being returned (https://github.com/spiritedmedia/spiritedmedia/issues/2455) etc.

After looking at how Humanmade designed Tachyon they use the API Gateway service to connect a Lambda function to the Internet. Our CDN points to a CloudFront distribution which points to the API Gateway which passes the request to a Lambda function, which queries S3 for the object which then returns the request using the reverse path. The problem with this is the API Gateway is a bottleneck. It requires binary responses (images, PDFs, videos) be base64 encoded strings which the API Gateway then decodes. The API Gateway service also has a limit to the total size of the payload, somewhere around 4-5MB. Payloads bigger than this result in a server error and are thus not served for the request.

Sine Tachyon was originally released Amazon introduced Lambda@Edge which lets you run a Lambda function at different points of a request to their CDN, CloudFront. With Lambda@Edge we can listen for an event and modify the location of the request to the origin server that fetches the asset. Instead of trying to make Lambda handle the request, we can use a Lambda function to change the logic of the request and rely on CloudFront to do the rest of the work of actually returning the response to that request.

How it works is we listen for when CloudFront needs to make a request to the origin server because it doesn't have a cached copy of the response to the URL being requested. If the request isn't for an image, then we bail on the Lambda function and let CloudFront continue to do its thing. But if the image is an image and has query strings that we recognize for manipulation then we can check if the image has already been processed and return that cached response. Otherwise we can pull down the original asset from S3, modify if (resize, manipulate colors, flip or flop dimensions etc.), save the result in a new directory on the S3 bucket, and tell CloudFront to use the new location to service the request. The result of this work is our own project Tachyon@Edge at https://github.com/spiritedmedia/tachyon-edge/

And as of 2/23/2018 it is now live on our own infrastructure serving requests.

[montchr]

[montchr]

Should we make this plugin open source? Seems like something that would be useful to others.