DanielHindi / aws-s3-zipper

takes an amazon s3 bucket folder and zips it for streaming or serializes to a file
119 stars 74 forks source link

cannot run with nodejs.12X environment #44

Open murphyzhang613 opened 4 years ago

murphyzhang613 commented 4 years ago

we have a demo lambda which invoke the 'aws-s3-zipper' running against Nodejs6.10, and it's working. Since the nodejs6.10 is end of life. After i export the demo lambda to another environment with Nodejs12.X or Nodejs10.0, both of them return below error. return message: { "errorType": "Runtime.ExitError", "errorMessage": "RequestId: 05ebdd8f-3860-47e1-9ccb-c88e08b6bcf0 Error: Runtime exited with error: signal: aborted (core dumped)" }

detail log: START RequestId: 05ebdd8f-3860-47e1-9ccb-c88e08b6bcf0 Version: $LATEST /var/lang/bin/node[8]: ../src/node_contextify.cc:649:static void node::contextify::ContextifyScript::New(const v8::FunctionCallbackInfo&): Assertion `args[1]->IsString()' failed. 1: 0x55788121a0f3 node::Abort() [/var/lang/bin/node] 2: 0x55788121a18c [/var/lang/bin/node] 3: 0x55788120baf6 node::contextify::ContextifyScript::New(v8::FunctionCallbackInfo const&) [/var/lang/bin/node] 4: 0x55788141cf9d [/var/lang/bin/node] 5: 0x55788141dd8c v8::internal::Builtin_HandleApiCall(int, unsigned long, v8::internal::Isolate) [/var/lang/bin/node] 6: 0x557881c81619 [/var/lang/bin/node] END RequestId: 05ebdd8f-3860-47e1-9ccb-c88e08b6bcf0 REPORT RequestId: 05ebdd8f-3860-47e1-9ccb-c88e08b6bcf0 Duration: 5073.67 ms Billed Duration: 5100 ms Memory Size: 128 MB Max Memory Used: 112 MB Init Duration: 120.55 ms

DanielHindi commented 4 years ago

Have you tried creating logs and viewing how far it gets in AWS Cloud Watch? This will at least show you how far it gets before failing. Also, your configuration needs to be Lambda friendly. An example is where you are dumping temp files and such. There are a few restrictions you need to be mindful of.

murphyzhang613 commented 4 years ago

exports.handler = function(event, context,callback) {

console.log('start');
const S3Zipper = require('aws-s3-zipper');
console.log('end'); 

} I tried the script as above, which still return the same error. Response: { "errorType": "Runtime.ExitError", "errorMessage": "RequestId: 52197942-725f-41fa-9e62-84fb114798be Error: Runtime exited with error: signal: aborted (core dumped)" }

Request ID: "52197942-725f-41fa-9e62-84fb114798be"

Function Logs: START RequestId: 52197942-725f-41fa-9e62-84fb114798be Version: $LATEST 2020-03-12T01:48:35.802Z 52197942-725f-41fa-9e62-84fb114798be INFO start 2020-03-12T01:48:40.580Z 52197942-725f-41fa-9e62-84fb114798be INFO ContextifyScript=function ContextifyScript() { [native code] } /var/lang/bin/node[7]: ../src/node_contextify.cc:635:static void node::contextify::ContextifyScript::New(const v8::FunctionCallbackInfo&): Assertion `args[1]->IsString()' failed. 1: 0x55e9a81291d3 node::Abort() [/var/lang/bin/node] 2: 0x55e9a81292cc [/var/lang/bin/node] 3: 0x55e9a81617ac node::contextify::ContextifyScript::New(v8::FunctionCallbackInfo const&) [/var/lang/bin/node] 4: 0x55e9a8419143 v8::internal::Builtin_HandleApiCall(int, v8::internal::Object*, v8::internal::Isolate) [/var/lang/bin/node] 5: 0x26d43b5be1d END RequestId: 52197942-725f-41fa-9e62-84fb114798be REPORT RequestId: 52197942-725f-41fa-9e62-84fb114798be Duration: 5514.55 ms Billed Duration: 5600 ms Memory Size: 128 MB Max Memory Used: 119 MB Init Duration: 151.60 ms
RequestId: 52197942-725f-41fa-9e62-84fb114798be Error: Runtime exited with error: signal: aborted (core dumped) Runtime.ExitError

this is all of the error message i got. Thanks very much

andersryanc commented 3 years ago

Seems like this might be related to the "Primordials" issue related to graceful-fs and Node 12+. This issue might be related?

https://github.com/andrewrk/node-s3-client/issues/228

I was able to get this working locally, by making a copy of your index.js and just using it directly with one small modification:

// change `s3` to `@auth0/s3`
var s3 = require('@auth0/s3');

The @auth0/s3 fork is mentioned in the above issue and has the newer graceful-fs dependency.

There is also a StackOverflow question related to this.