jeremydaly / lambda-api

Lightweight web framework for your serverless applications
https://serverless-api.com
MIT License
1.41k stars 125 forks source link

Sending JSON with res.send vs sending images with res.sendFile #256

Closed RWhite-EDR closed 4 months ago

RWhite-EDR commented 6 months ago

I have a route that either sends data gathered from a database or an image depending on an extension passed to the route:

/api/states/ma/cities/lenox           returns JSON data about Lenox, MA
/api/states/ma/cities/lenox.png   returns an image

For the JSON data, I'm gathering the results and calling: return res.status(200).json(data);

For the Image data, I'm gathering the image from S3 and calling: return res.status(200).sendFile('s3:///ma/lenox.png');

The sendFile on the image is only working if compression is turned off for the app:

const api = require('lambda-api')( { base: '/api/edr20/citydirectory',
                                     compression: false,
                                     logger: { level: 'warn',
                                               stack: process.env.STAGE_ENV === 'DEV' ? true : false } } );

I've been debugging through the response.js file and I'm not really sure what is going on at this point.

At first I thought that this was a difference between API Gateway and ALB (which I'm using), but I'm seeing the same issue in serverless-offline.

RWhite-EDR commented 5 months ago

There is something wrong with compression and returning binary files. If I turn compression off, I get significantly larger JSON response and successfully return images. If I turn compression on, I get significantly smaller JSON response and slightly larger images that aren't recognized.

Is it possible to turn compression off for specific routes? It appears to be only application wide with false, true, or an array of compression methods.

naorpeled commented 5 months ago

There is something wrong with compression and returning binary files. If I turn compression off, I get significantly larger JSON response and successfully return images. If I turn compression on, I get significantly smaller JSON response and slightly larger images that aren't recognized.

Is it possible to turn compression off for specific routes? It appears to be only application wide with false, true, or an array of compression methods.

Hey @RWhite-EDR, I'll try to take a look at this tomorrow 🙏

RWhite-EDR commented 4 months ago

@naorpeled Thanks for your offer, but let's hold off a bit.

I think I'm running into an ALB limit on the response size. It isn't well documented, but I'm seeing that the header and body must be under 1 MB and I'm not seeing that this is configurable.

I'll update when I've learned more.

RWhite-EDR commented 4 months ago

@naorpeled I was able to confirm that I was having an ALB response size issue. I have switched to API Gateway. My understanding is that the response sizes allowed are as follows: ALB (< 1MB) < Lambda (<6MB) < API Gateway (<10MB)

I'm going to check and see if serverless-offline allows some sort of set'able limit which would have helped me catch this.