Cannot read property 'build' of null - aws-kinesis-agg/lib/common.js

Strydom commented 4 years ago

Hello,

I'm having issues using this library in a Typescript project that is being webpacked. I'm able to run the code in the TS package (blacksmith-kinesis-mapper) where i use the aws-kinesis-agg library and when I run my integration tests before packaging the Node Lambda (blacksmith-foundry-dynamo-listener) that pulls in the TS package.

The issue arrises when I package the lambda code up using webpack. I get the error in AWS and when i run the code locally with lambda-local.

error: {
    "errorMessage": "Cannot read property 'build' of null",
    "errorType": "TypeError",
    "stackTrace": [
        "loadBuilder (/Users/chstrydom/Documents/GitHub/blacksmith-foundry-dynamo-listener/build/index.js:5779:17)",
        "Object.<anonymous> (/Users/chstrydom/Documents/GitHub/blacksmith-foundry-dynamo-listener/build/index.js:5787:26)",
        "Object../node_modules/aws-kinesis-agg/lib/common.js (/Users/chstrydom/Documents/GitHub/blacksmith-foundry-dynamo-listener/build/index.js:5797:30)",
        "__webpack_require__ (/Users/chstrydom/Documents/GitHub/blacksmith-foundry-dynamo-listener/build/index.js:20:30)",
        "Object../node_modules/aws-kinesis-agg/lib/kpl-deagg.js (/Users/chstrydom/Documents/GitHub/blacksmith-foundry-dynamo-listener/build/index.js:6316:16)",
        "__webpack_require__ (/Users/chstrydom/Documents/GitHub/blacksmith-foundry-dynamo-listener/build/index.js:20:30)",
        "Object../node_modules/aws-kinesis-agg/index.js (/Users/chstrydom/Documents/GitHub/blacksmith-foundry-dynamo-listener/build/index.js:5719:34)",
        "__webpack_require__ (/Users/chstrydom/Documents/GitHub/blacksmith-foundry-dynamo-listener/build/index.js:20:30)",
        "Object../node_modules/blacksmith-kinesis-mapper/lib/deaggregate.js (/Users/chstrydom/Documents/GitHub/blacksmith-foundry-dynamo-listener/build/index.js:23714:43)",
        "__webpack_require__ (/Users/chstrydom/Documents/GitHub/blacksmith-foundry-dynamo-listener/build/index.js:20:30)"
    ]
}

The stack trace seems to point to https://github.com/awslabs/kinesis-aggregation/blob/master/node/lib/common.js#L43

I'm not sure If it's an issue with my project or if you need to do something to prevent webpack from treeshaking needed files etc?

webpack: 4.43.0
typescript: 3.7.5
serverless-webpack: 5.3.2

IanMeyers commented 4 years ago

I would guess that the builder is null because the Protobuf library isn't working properly, or there is no Protobuf being included in the distribution. I'm afraid I don't know enough about how the typescript binary might be created to comment further.

Strydom commented 4 years ago

When i run the build I do get this warning:

WARNING in ./node_modules/bytebuffer/dist/bytebuffer-node.js
Module not found: Error: Can't resolve 'memcpy' in '/Users/chstrydom/Documents/GitHub/blacksmith-foundry-dynamo-listener/node_modules/bytebuffer/dist'
 @ ./node_modules/bytebuffer/dist/bytebuffer-node.js
 @ ./node_modules/protobufjs/dist/protobuf.js
 @ ./node_modules/aws-kinesis-agg/lib/common.js
 @ ./node_modules/aws-kinesis-agg/lib/kpl-deagg.js
 @ ./node_modules/aws-kinesis-agg/index.js
 @ ./node_modules/blacksmith-kinesis-mapper/lib/deaggregate.js
 @ ./node_modules/blacksmith-kinesis-mapper/lib/KinesisMapper.js
 @ ./src/index.ts

Strydom commented 4 years ago

Can you let me know when the new version is published please?

IanMeyers commented 4 years ago

https://www.npmjs.com/package/aws-kinesis-agg/v/4.1.0

Strydom commented 4 years ago

Could you push a hotfix please? We forgot to update this https://github.com/awslabs/kinesis-aggregation/blob/master/node/package.json#L30 so the .json version hasn't been published

awslabs / kinesis-aggregation

Cannot read property 'build' of null - aws-kinesis-agg/lib/common.js #99