dphang
commented
3 years ago The other solution might be always optimizing it and allowing CloudFront to cache the responses. That way we avoid using lambda memory and depend on CloudFront functionality instead.
It already has cache control headers, so each image size/quality combination will only be optimized once per CloudFront edge location, until the cache is purged. It should not calling the image Lambda more than that, so I don't think the cost issue is that big, although it can be optimized. But yes, it can be optimized further since this still involves 200+ Lambda optimization invocations + 200+ S3 invocations (to retrieve image to be optimized) per combination. Why 200? Assuming there are ~200 Lambda CloudFront edge locations from what I recall.
If we store optimized images on S3, yes we can reduce Lambda execution time by skipping the sharp optimization code, though we should ensure we version our images using the value of NEXT_BUILD_ID, as there can be problems when you have same file names across two app versions, with different content (e.g you updated the logo). Usually the solution to this is to version your image names, e.g adding an MD5 hash of the image data to the filename. But if user does not do that, doing this way would be good to ensure no conflicts. Though here there will still be requests to S3 (again ~200 requests, but for optimized image instead of original), though Lambda execution time is reduced. So I think the cost savings should be moderate but not that much.
What could really help is if we instead store the original image and possibly also pre-optimized images in the Lambda bundle itself, then we would only have a fast Lambda call (not even hitting S3/image optimizer code - simply serving an image from the bundle). Though for app with many images, it might not be feasible to do so due to Lambda 50 MB limitation.
yasserf
iDVB
When a user uploads an image, instead of it being written to the lambda file system (seen here) we can instead use an s3 bucket (possibly even the one we normally use for deployment).
Since we deploy on edge + not really knowing how long lambda tasks last + using up memory in a lambda function doesn't scale well financially (https://aws.amazon.com/lambda/pricing/). It also means whenever doing a CloudFront cache bust we end up having to redo all the images.
We can also use the expiration logic on S3 objects to automatically clean up objects that expire similarly to nextJS.
So we could move to this sort of setup instead similar infrastructure instead.
That way the solution would work globally which is quite nice instead of being duplicated per edge server.
The other solution might be always optimizing it and allowing CloudFront to cache the responses. That way we avoid using lambda memory and depend on CloudFront functionality instead.