sst / ion

SST v3
https://sst.dev
MIT License
1.56k stars 200 forks source link

nodejs-runtime: undici fetch error -- unable to use sst dev on requests > 100kb #655

Open musubipapi opened 2 months ago

musubipapi commented 2 months ago

Error I'm getting when running sst dev on a POST multipart form lambda using APIGatewayV2

fetch failed
|  ↳ at node:internal/deps/undici/undici:12502:13
|  ↳ at async error (file:///Users/user/Desktop/projects/project/.sst/platform/dist/nodejs-
runtime/index.js:11:3)
|  ↳ at async file:///Users/user/Desktop/projects/project/.sst/platform/dist/nodejs-
runtime/index.js:91:5

After investigating a bit further the issue comes from the .json() returning an Unexpected token... invalid JSON etc etc. inside of the sst nodejs-runtime/index.js

 const result = await fetch(AWS_LAMBDA_RUNTIME_API + `/runtime/invocation/next`);
    context = {
      awsRequestId: result.headers.get("lambda-runtime-aws-request-id") || "",
      invokedFunctionArn: result.headers.get("lambda-runtime-invoked-function-arn") || "",
      getRemainingTimeInMillis: () => Math.max(Number(result.headers.get("lambda-runtime-deadline-ms")) - Date.now(), 0),
      identity: (() => {
        const header = result.headers.get("lambda-runtime-cognito-identity");
        return header ? JSON.parse(header) : undefined;
      })(),
      clientContext: (() => {
        const header = result.headers.get("lambda-runtime-client-context");
        return header ? JSON.parse(header) : undefined;
      })(),
      functionName: process.env.AWS_LAMBDA_FUNCTION_NAME,
      functionVersion: process.env.AWS_LAMBDA_FUNCTION_VERSION,
      memoryLimitInMB: process.env.AWS_LAMBDA_FUNCTION_MEMORY_SIZE,
      logGroupName: result.headers.get("lambda-runtime-log-group-name") || "",
      logStreamName: result.headers.get("lambda-runtime-log-stream-name") || "",
      callbackWaitsForEmptyEventLoop: {
        set value(_value) {
          throw new Error("`callbackWaitsForEmptyEventLoop` on lambda Context is not implemented by SST Live Lambda Development.");
        },
        get value() {
          return true;
        }
      }.value,
      done() {
        throw new Error("`done` on lambda Context is not implemented by SST Live Lambda Development.");
      },
      fail() {
        throw new Error("`fail` on lambda Context is not implemented by SST Live Lambda Development.");
      },
      succeed() {
        throw new Error("`succeed` on lambda Context is not implemented by SST Live Lambda Development.");
      }
    };
    request = await result.json();

It looks like the result's body from AWS_LAMBDA_RUNTIME_API is fetching an incomplete response/response that's cut off (~100kb mark)

Example response:

{"version":"2.0","routeKey":"POST /parse-resume","rawPath":"/parse-resume","rawQueryString":"","headers":{"accept":"*/*","content-length":"114640","content-type":"multipart/form-data; boundary=X-INSOMNIA-BOUNDARY","host":"nkdhuh1bkb.execute-api.us-east-1.amazonaws.com","user-agent":"insomnia/9.2.0","x-amzn-trace-id":"Root=1-668e5970-7e4fc31d36dce94d177140fc","x-forwarded-for":"218.153.58.96","x-forwarded-port":"443","x-forwarded-proto":"https"},"requestContext":{"accountId":"992382568513","apiId":"nkdhuh1bkb","domainName":"nkdhuh1bkb.execute-api.us-east-1.amazonaws.com","domainPrefix":"nkdhuh1bkb","http":{"method":"POST","path":"/parse-resume","protocol":"HTTP/1.1","sourceIp":"218.153.58.96","userAgent":"insomnia/9.2.0"},"requestId":"asLpriyGIAMEYDg=","routeKey":"POST /parse-resume","stage":"$default","time":"10/Jul/2024:09:50:40 +0000","timeEpoch":1720605040834},"body":"LS1YLUlOU09NTklBLUJPVU5EQVJZDQpDb250ZW50LURpc3Bvc2l0aW9uOiBmb3JtLWRhdGE7IG5hbWU9ImZpbGUiOyBmaWxlbmFtZT0iTWFyaWEgUmVzdW1lLnBkZiINCkNvbnRlbnQtVHlwZTogYXBwbGljYXRpb24vcGRmDQoNCiVQREYtMS4zCiXE5fLl66fzoNDExgo0IDAgb2JqCjw8IC9MZW5ndGggNSAwIFIgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCngBtL1LzyQ5kp67/37FtxYw1eHucQUas5geYaDFABKmAa2P6kxrIHQdYNT/HzjPaySN5k4yMryaWQ10Rr7pNOOddqPxP7//x/d/fl++H6/1+7fv+7Lar7/ar8svF/13f3z/lS/i3/7j+3/+l+//T6AX+b///vU/gf7wp78t37/+7Xux//3t199B+i/f/+NrUpW+/+fXrCp9/+fXlF76ppe+rt//+v2Hf6Gn/vffUo/96d+sMy/f//YnhuMfLr88bgnQj9v1+suyXr8f2/LLwt9//U3/9stNA7Zuv9yvj+VLf/3r978xmnk8t/WXjQ9WylwvN8rq73/1v18ptz6uVxva8Jf/+OqMrOrsI3v55XW9LfCPf6SB3vP8+pjnN0P+41n49dEsPNR1uzKlL7/EP/7269fyeNCjz43+u//yeD3u9NTyuP7yfETsrxF7/HK78p0wL5sw6qWyLb2lQ8+xQM/LBnqM9LF+92dLr2KBnpet9PiupXfr0HOs0qtlA71bh97aoedYoOdlA721pXd7tfQqFuh52UqP75r23u4deo5VerVsoHfv0Ns69BwL9LxsoLd16F069BwL9LxsoHdp6V0fLb2KBXpettLju6b/2IWa9VGxSq+WDfR8jtf1dvW1kDHWUcUCPS8b6Pkcr/S2zvqoWKDnZSs9vmvau/laqPWrWKVXywZ6PsdD/XwtBHqOBXpeNtDzOV7prZ31U[does not complete]

My guess is there's some kind of data body size limit at ~100kb when running sst dev

goto3 commented 2 months ago

I have the same issue

chrisandrewcl commented 2 months ago

Not sure if it is the same thing, but I was getting a similar error which cleared after restarting sst dev:

|  fetch failed
|  ↳ at node:internal/deps/undici/undici:12502:13
|  ↳ at async file:///home/chrisandrew/workspace/project/.sst/platform/dist/nodejs-runtime/index.js:51:2
rauthf commented 1 month ago

I have the same issue. Can't invoke anymore.