joshuaflanagan / serverless-ruby-package

serverless plugin to package ruby gems
39 stars 11 forks source link

Native extensions built on linux are not working properly #2

Closed joshuaflanagan closed 5 years ago

joshuaflanagan commented 5 years ago

When developing on Linux, the gem native extensions are not compiled using the docker container that emulates the lambda environment - they are compiled using the local Linux environment.

I've observed that the compilation output is different. For one, in my local ubuntu environment, the extensions are compiled to vendor/bundle/ruby/2.5.0/extensions/x86_64-linux/2.5.0/, whereas lambda expects them at vendor/bundle/ruby/2.5.0/extensions/x86_64-linux/2.5.0-static/. I solved this by creating a symlink so that the package will include them at the 2.5.0-static location.

That was successful in including the correct files, however, I suspect the output files were not created as needed. When I deploy the package created on Ubuntu, the demo_service endpoint fails with this error:

$> curl https://deployedendpoint.execute-api.us-east-1.amazonaws.com/dev/
{"alpha":"first","beta":"second","redis_version":"4.1.0",
"error_class":"LoadError",
"error_message":"libruby-2.5.so.2.5: cannot open shared object file: No such file or directory - /var/task/vendor/bundle/ruby/2.5.0/gems/http_parser.rb-0.6.0/lib/ruby_http_parser.so"
}

It is failing at the part that uses the http gem, which relies on a native extension.

I suspect that in my local Ubuntu system, the extensions are being built with shared/dynamic linking, while the docker build container compiles them with static linking. Need to figure out how to force static compiling locally, or else always build extensions using the docker container, even when developing on Linux.

lwoodson commented 5 years ago

Amazon linux is the OS used in the lambda execution environment. I would experiment with using that in your docker container to compile the dependencies. Its available from docker-hub https://hub.docker.com/_/amazonlinux.

There is also this knowledge center article that might provide some insight https://aws.amazon.com/premiumsupport/knowledge-center/lambda-linux-binary-package/