iliana / rust-crowbar

Wrapper to simplify writing AWS Lambda functions in Rust (using the Python execution environment)
https://docs.rs/crowbar
Apache License 2.0
197 stars 16 forks source link

[builder] Unable to import module 'liblambda': /lib64/libcrypto.so.10: version `OPENSSL_1.0.2' not found (required by /var/task/liblambda.so) #20

Closed iliana closed 5 years ago

iliana commented 6 years ago

Amazon Linux 2017.09 ships with OpenSSL 1.0.2; Lambda is still on 1.0.1.

Making things more difficult, Python 3.6 was first released in Amazon Linux 2017.09, so there's not a build against OpenSSL 1.0.1, so we can't even downgrade openssl-devel.

Including libcrypto.so.10 in the zip doesn't seem to help either; I think /var/task is lower precedence than system library paths in Lambda.

ramn commented 6 years ago

Any progress on this or any ideas?

iliana commented 6 years ago

Not currently.

If you're interested in spending some time to try and figure things out, the lambci docker-lambda images might work okay (apparently aws-sam-local uses them).

ramn commented 6 years ago

It works! Thanks. I just tried manually, perhaps that image can be used by crowbar.

ramn commented 6 years ago

In the crowbar builder Dockerfile, I updated the FROM line to this:

FROM lambci/lambda:build-python3.6

And in my project, in Cargo.toml I added this:

python3-sys = { version = "0.1.3", features = ["python-3-4"] }

Where the feature "python-3-4" is the important part. However, with that feature enabled I can't build locally (on my mac, that is, not using the docker container). Perhaps that can be solved? Can we enable this feature only in the docker run?

ramn commented 6 years ago

It also works if I set python3-sys = { version = "0.1.3", features = ["python-3-4"] } in rust-crowbar/Cargo.toml. Just need to be able to toggle this as a feature when running cargo build.

ramn commented 6 years ago

So I found a way of building locally with default python binding but override to the python3.4 when building in the crowbar builder docker container. I'll do a PR.

iliana commented 6 years ago

python3-sys = { version = "0.1.3", features = ["python-3-4"] }

That is... super weird. Why is there even a python3.4 in that image...

ramn commented 6 years ago

Yes and I tried with python-3-5 which didn't work.

softprops commented 6 years ago

this is still an open issue. what is the recommended workaround? I'm getting the same error. not being able to make https requests is a bit debilitating

Unable to import module 'liblambda': /lib64/libcrypto.so.10: version `OPENSSL_1.0.2' not found (required by /var/task/liblambda.so)

END RequestId: 420f016d-d7ed-11e7-bcec-c986dbdea94e
REPORT RequestId: 420f016d-d7ed-11e7-bcec-c986dbdea94e  Duration: 15.37 ms  Billed Duration: 100 ms     Memory Size: 1024 MB    Max Memory Used: 22 MB
softprops commented 6 years ago

the work around @ramn mentioned worked for me.

nbigaouette-eai commented 6 years ago

The Docker image from lambci should be an (almost) identical environment than the official one. The reason is that they created the image by saving the whole filesystem to S3 using tar from inside a lambda! See https://github.com/lambci/docker-lambda#questions , specially the Wut, how? bullet point.

You can thus inspect the environment locally with docker run -it --rm lambci/lambda:build-python3.6 bash.

Python 3.6 is installed in /var/lang/bin/:

> which python3.6
/var/lang/bin/python3.6

but the one in /usr/bin is Python 3.4!

> ls -l /usr/bin/python3*
lrwxrwxrwx 1 root root   25 Sep 20 19:05 /usr/bin/python3 -> /etc/alternatives/python3
-rwxr-xr-x 3 root root 6864 Sep  1  2016 /usr/bin/python34
-rwxr-xr-x 3 root root 6864 Sep  1  2016 /usr/bin/python3.4
lrwxrwxrwx 1 root root   17 Sep 20 19:05 /usr/bin/python3.4-config -> python3.4m-config
-rwxr-xr-x 3 root root 6864 Sep  1  2016 /usr/bin/python3.4m
-rwxr-xr-x 1 root root  173 Sep  1  2016 /usr/bin/python3.4m-config
-rwxr-xr-x 1 root root 3288 Sep  1  2016 /usr/bin/python3.4m-x86_64-config
lrwxrwxrwx 1 root root   32 Sep 20 19:05 /usr/bin/python3-config -> /etc/alternatives/python3-config

This means that if one wants to use the python interpreter that is linked to the OS provided openssl (/lib64/libcrypto.so.1.0.1k), one has to use the OS python interpreter /usr/bin/python3 which is... Python 3.4.

That explains the python-3-4 feature required for the python3-sys crate.

iliana commented 5 years ago

I think this is fixed now that we refer people to https://github.com/naftulikay/docker-crowbar?