Node.js Async Handler not working as expected

[chumaumenze]

I have a lambda function that uss async/await. The function works well locally using lambci/lambda:nodejs12.x image. However, on AWS Lambda, the function doesn't seem to work as expect. It returns null and errors.

I can't seem to figure it out but my guess is that it's an async problem. Currently, this is what my handler looks like:

exports.main = async (event, context) => {
  let resp = {
    status: "success",
    message: "PDF generated successfully.",
    data: {
      pdf: null,
      log: null,
    },
  };
  let data;
  switch (event.type) {
    case "url": {
      data = await html2pdf.fromURL(event.value);
      break;
    }
    case "string": {
      if (["pug", "html"].includes(event.format)) {
        data = await html2pdf.fromTemplate(event.value, event.format);
      } else {
        data = { rv: null, err: "Invalid input format specified." };
      }
      break;
    }
    default: {
      data = { rv: null, err: "Invalid type specified." };
    }
  }
  if (data.rv) {
    resp.data.pdf = `data:application/pdf;base64,${data.rv.toString("base64")}`;
  } else {
    resp.status = "failure";
    resp.message = data.err;
  }
  return resp;
};

Full code can be found here: https://gist.github.com/chumaumenze/b15a2d64b95294a8bac3218384a6c9a7

[chumaumenze]

With the code in the gist, I always get the value null.

When I change the return value of main to return new Promise((res, rej) => res(resp));, I keep getting errors such as:

ERROR   TypeError: Cannot read property 'newPage' of undefined
    at /var/task/html2pdf.js:177:33
    at HTML2PDF.getBrowser (/var/task/html2pdf.js:74:20)

ERROR   Error: Protocol error (Target.createTarget): Target closed.
    at /opt/nodejs/node_modules/puppeteer/lib/Connection.js:74:56
    at new Promise (<anonymous>)
    at Connection.send (/opt/nodejs/node_modules/puppeteer/lib/Connection.js:73:12)
    at Browser._createPageInContext (/opt/nodejs/node_modules/puppeteer/lib/Browser.js:174:47)
    at BrowserContext.newPage (/opt/nodejs/node_modules/puppeteer/lib/Browser.js:367:26)
    at Browser.newPage (/opt/nodejs/node_modules/puppeteer/lib/Browser.js:166:33)
    at Browser.<anonymous> (/opt/nodejs/node_modules/puppeteer/lib/helper.js:112:23)
    at /var/task/html2pdf.js:187:33
    at HTML2PDF.getBrowser (/var/task/html2pdf.js:61:20)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
    at async HTML2PDF.fromTemplate (/var/task/html2pdf.js:185:12)
    at async Runtime.exports.main [as handler] (/var/task/html2pdf.js:223:16)

ERROR Error: Protocol error (Target.setDiscoverTargets): Target closed.

[mhart]

I notice you're using chromium, so it might be something obscure in the Lambda environment that's getting in the way. Specifically, there are some devices that are hard to emulate exactly in the docker environment that chromium might rely on – like audio or video drivers or similar.

See if you get similar results if you add the follow volume mappings (to try and replicate these devices being missing):

docker ...
  -v /dev/null:/dev/fd/dummy \
  -v /dev/null:/dev/shm \
  -v /dev/null:/dev/mqueue \
  -v /dev/null:/dev/core \
  -v /dev/null:/dev/tty \
  ...

[chumaumenze]

@mhart I mounted the volumes as suggested.

find ./ -name "*.js*" -not -path "*node_modules*" | \
    entr -cr docker run --rm \
    -e PAGE_TIMEOUT=30 -e DOCKER_LAMBDA_STAY_OPEN=1 \
    -e AWS_LAMBDA_FUNCTION_NAME=html2pdf \
    -p 9001:9001 -v /dev/null:/dev/fd/dummy  
    -v /dev/null:/dev/shm -v /dev/null:/dev/mqueue \
    -v /dev/null:/dev/core -v /dev/null:/dev/tty \
    -v "$PWD":/var/task:ro,delegated \
    -v "$PWD"/nodejs/node_modules:/opt/nodejs/node_modules:ro,delegated \
    lambci/lambda:nodejs12.x html2pdf.main

Invoke function

aws lambda invoke \
    --endpoint http://localhost:9001 \
    --no-sign-request \
    --function-name=html2pdf \
    --invocation-type=RequestResponse \
    --payload $(echo '{"type": "string", "format": "pug", "input": "h1 Hello World!\np My name is Chuma Umenze"}' | base64 ) \
    output.json

Output

START RequestId: 3bb6e50c-346e-1412-355d-9e7badf56785 Version: $LATEST
2020-06-16T11:52:34.115Z        3bb6e50c-346e-1412-355d-9e7badf56785    ERROR   Unhandled Promise Rejection  
   {"errorType":"Runtime.UnhandledPromiseRejection","errorMessage":"Error: Page crashed!","reason":{"errorType":"Error","errorMessage":"Page crashed!","stack":["Error: Page crashed!","
    at Page._onTargetCrashed (/opt/nodejs/node_modules/puppeteer/lib/Page.js:215:24)","
    at CDPSession.<anonymous> (/opt/nodejs/node_modules/puppeteer/lib/Page.js:123:56)","
    at CDPSession.emit (events.js:310:20)","
    at CDPSession._onMessage (/opt/nodejs/node_modules/puppeteer/lib/Connection.js:200:12)","
    at Connection._onMessage (/opt/nodejs/node_modules/puppeteer/lib/Connection.js:112:17)","
    at WebSocket.<anonymous> (/opt/nodejs/node_modules/puppeteer/lib/WebSocketTransport.js:44:24)","
    at WebSocket.onMessage (/opt/nodejs/node_modules/ws/lib/event-target.js:120:16)","
    at WebSocket.emit (events.js:310:20)","
    at Receiver.receiverOnMessage (/opt/nodejs/node_modules/ws/lib/websocket.js:789:20)","
    at Receiver.emit (events.js:310:20)","
    at Receiver.dataMessage (/opt/nodejs/node_modules/ws/lib/receiver.js:422:14)","
    at Receiver.getData (/opt/nodejs/node_modules/ws/lib/receiver.js:352:17)","
    at Receiver.startLoop (/opt/nodejs/node_modules/ws/lib/receiver.js:138:22)","
    at Receiver._write (/opt/nodejs/node_modules/ws/lib/receiver.js:74:10)","
    at doWrite (_stream_writable.js:403:12)","
    at writeOrBuffer (_stream_writable.js:387:5)"]},"promise":{},"stack":["Runtime.UnhandledPromiseRejection: Error: Page crashed!","
    at process.<anonymous> (/var/runtime/index.js:35:15)","
    at process.emit (events.js:310:20)","
    at processPromiseRejections (internal/process/promises.js:209:33)","
    at processTicksAndRejections (internal/process/task_queues.js:98:32)"]}
END RequestId: 3bb6e50c-346e-1412-355d-9e7badf56785
REPORT RequestId: 3bb6e50c-346e-1412-355d-9e7badf56785  Duration: 61021.36 ms   Billed Duration: 61100 ms       Memory Size: 1536 MB    Max Memory Used: 257 MB 

[mhart]

So that looks like it's reproducing what happens on Lambda, yeah?

[chumaumenze]

The last error is not what I got on Lambda. However, on Lambda, the issue was with the connection between puppeteer and the browser (chrome-aws-lambda). I was able to resolve it by passing some chrome-aws-lambda.args to puppeteer.

Although, I expected docker-lambda to throw the same exception as on AWS Lambda — to fully emulate the lambda environment. Given that this error was not raised in local env but occurred on Lambda, do you think this might an issue on docker-lambda?

[mhart]

As I said in https://github.com/lambci/docker-lambda/issues/285#issuecomment-644432714 – it's hard when devices are involved to get exactly the same error. For example, /dev/shm doesn't exist in Lambda, but it does in docker (any image – not just docker-lambda).