Function URL invoke hangs when the response stream is empty

[paya-cz]

I have a lambda with response streaming enabled. Invoked via Function URL. I am trying to return a status code with no response body. I have reduced my code to the below sample to reproduce the issue:

// File: index.cjs
// Handler: index.handler

const pipeline = require("util").promisify(require("stream").pipeline);
const { Readable } = require("stream");

exports.handler = awslambda.streamifyResponse(async (event, responseStream, context) => {
  responseStream = awslambda.HttpResponseStream.from(responseStream, {
    statusCode: 403,
  });

  await pipeline(
    Readable.from(Buffer.alloc(0)),
    responseStream,
  );
});

I can see the HTTP headers are sent correctly and received by the client. But the connection remains open for up to 16 minutes. It seems like the runtime expects a response body, and if there is none, it just hangs and does not close the connection.

What's even stranger is that I have tested this in 2 different accounts. One of the accounts has been created ages ago, and the code above actually works fine as one would expect. The other account is newer (and it's where that code is supposed to run), and it just hangs and doesn't close the connection. Region is us-east-1.

[H4ad]

Call responseStream.end should solve your issue, or

pipeline(
    Readable.from(Buffer.alloc(0)),
    responseStream,
    { end: true },
)

[paya-cz]

Call responseStream.end should solve your issue, or

I believe pipeline automatically calls .end(), because end: true is the default. Nevertheless, I added both of your suggestions to the code:

const pipeline = require("util").promisify(require("stream").pipeline);
const { Readable } = require("stream");

exports.handler = awslambda.streamifyResponse(async (event, responseStream, context) => {
  responseStream = awslambda.HttpResponseStream.from(responseStream, {
    statusCode: 403,
  });

  console.log('before pipeline');

  await pipeline(
    Readable.from(Buffer.alloc(0)),
    responseStream,
    { end: true },
  );

  responseStream.end();

  console.log('after pipeline');
});

In lambda logs, I can see both messages before pipeline and after pipeline just as you would expect. No errors are thrown, the lambda execution ends within milliseconds after the after pipeline message. The entire execution takes fraction of a second. Nevertheless, when accessing Lambda URL via browser or Postman, the headers are sent but the connection hangs open - even though lambda stopped its execution long time ago.

[H4ad]

Hm... now I remembered that I already had an issue like this in my lib: https://github.com/H4ad/serverless-adapter/blob/250221f3e89a08d28d48c1258768175b5d176b15/src/handlers/aws/aws-stream.handler.ts#L307-L311

You need at least to call once .write, even with empty text.

This is probably a bug.

[paya-cz]

I have tested adding the empty text .write to my code but it didn't really make any difference. I am returning status code 403, the bug might also be status-code dependent I suppose.

[H4ad]

Strangely, this is supposed to solve the issue.

You can try my lib, which is basically a port of serverless-express, I already tried to handle all those cases, and if the bug is still happening, open an issue on my lib and I can try to investigate for you.

[paya-cz]

I just tested my code again (not @h4ad/serverless-adapter) and I don't have a problem with other status codes such as 304 or 204. But the 403 behaves as I explained. Must be an AWS bug.

[paya-cz]

I suppose AWS infrastructure Function URL uses code similar to this:

if (lambdaResponseStatusCode >= 400 && lambdaResponseStatusCode < 600) {
    await fetchResponse();
}

end();

So if there is an HTTP status code indicating error, the Function URL hangs if there is no response body because they assume error response must include response body. That's my theory at least.