Closed christian2345 closed 4 years ago
Parquetjs works well on AWS Lambda. However since lambda runs on linux you might have to build parquetjs (which include thrift c++ modules that have to be compiled) in a linux docker or vm.
If you go with Docker: amazonlinux:2017.03
matches the Lambda execution environment.
thanks, I'll give it a try - may have to reconsider since win10 is only the development platform for an AWS application
Thanks @vweevers yes this is how I build stuff that has to be compiled. @christian2345 Visual Studio (Windows) has plugin to publish to aws lambda, but that is not the only delivery mechanism. You can go directly to aws lambda and upload a zip file or use the aws-cli:
aws lambda update-function-code --function-name my-function --zip-file fileb://mycode.zip
@ZJONSSON Are there any additional steps you have to take in order to make it work on Lambda? I have a Docker container running amazon linux with node 8.10, running npm install
on my project which includes parquetjs
. After zipping and uploading, my Lambda still gives me an error like before:
"errorMessage": "/var/lang/lib/libstdc++.so.6: version GLIBCXX_3.4.21' not found (required by /var/task/node_modules/lzo/build/Release/node_lzo.node)",
Is there anything I have to do like moving .so
files to a /lib
folder?
@aconanlai Amazon Linux 1 or Amazon Linux 2? You may need to install or update some packages in order to build native modules:
sudo yum install gcc gcc-c++ libgcc make
Sorry, I misread, didn't see you already successfully ran npm install
.
@vweevers thanks for the response. this is what i'm doing:
FROM amazonlinux:latest
add my files, etc
RUN yum install gcc44 gcc-c++ libgcc44 cmake wget tar zip gzip make -y
install node, etc
npm install
zip, upload, etc
(required by /var/task/node_modules/lzo/build/Release/node_lzo.node)
@aconanlai Seeing as the error is coming from lzo
, another native module, maybe try asking there.
@aconanlai, are you copying node_modules to docker image? Try adding node_modules to .dockerignore file?
@anton-yurchenko i'm only copying my lambda code and the package.json
.
i solved this by using images from here rather than the amazonlinux
image:
Hi there,
I have installed - with some difficulty - parquetjs under node v8.11.4 in win10/VisualStudio2017. The ultimate destination is a Lambda function under AWS. However, there are some errors during the load phase (fs.opensync(config.json) - which does not exist at the path provided). I am wondering if parquetjs can be used in this environment ... ?
parquetjs_install.txt