import pyvips throws exception for python 2.7 and 3.6 in AWS lambda (ZLIB_1.2.9 not found while loading libvips.so.42)

[munirhossain]

I'm writing a AWS Lambda function using pyvips python package. But it throws exception when import pyvips is executed. The error is as follows. I've zipped cffi, pycparser, pyvips and a lib directory which contains all the shared libraries of libvips for lambda build package. I've also set the environment variable LD_LIBRARY_PATH=./lib:LD_LIBRARY_PATH .The lambda.zip that I'm uploading to the AWS lambda can be found here.

cannot load library 'libvips.so.42': /lib64/libz.so.1: version `ZLIB_1.2.9' not found (required by ./lib/libpng16.so.16).  Additionally, ctypes.util.find_library() did not manage to locate a library called 'libvips.so.42': OSError
Traceback (most recent call last):
  File "/var/task/lambda.py", line 5, in lambda_handler
    import pyvips
  File "/var/task/pyvips/__init__.py", line 54, in <module>
    vips_lib = ffi.dlopen(_vips_libname)
  File "/var/task/cffi/api.py", line 141, in dlopen
    lib, function_cache = _make_ffi_library(self, name, flags)
  File "/var/task/cffi/api.py", line 802, in _make_ffi_library
    backendlib = _load_backend_lib(backend, libname, flags)
  File "/var/task/cffi/api.py", line 797, in _load_backend_lib
    raise OSError(msg)
OSError: cannot load library 'libvips.so.42': /lib64/libz.so.1: version `ZLIB_1.2.9' not found (required by ./lib/libpng16.so.16).  Additionally, ctypes.util.find_library() did not manage to locate a library called 'libvips.so.42'

END RequestId: af867599-5876-11e8-a3ca-ffec3ebba19d
REPORT RequestId: af867599-5876-11e8-a3ca-ffec3ebba19d  Duration: 459.45 ms Billed Duration: 500 ms     Memory Size: 128 MB Max Memory Used: 21 MB  

This may not be a problem of libvips or pyvips but some setting in AWS Lambda function that I don't know. Created an issue with AWS as well.

[jcupitt]

Hello, I'll have a go at a docker build on aws.

[jcupitt]

This seems to work:

https://github.com/jcupitt/docker-builds/tree/master/pyvips-aws

Though I don't know if that'll help you much.

[munirhossain]

I think docker would be fine since it manages all its libraries. In my case I'm running a Lambda function without docker. And the problem is that even though I provide all the necessary libraries in lib directory and set the environment path LD_LIBRARY_PATH=./lib. It seems Lambda in unable to load all the libraries from this lib directory. I've found a work around which is running another process with my correct configuration set for the library path and load pyvips in that process only. It seems to work. It is a work around and the down side is that you need to create another process and manage it. So it comes with all the regular downside of it. But for now it is working fine. I mainly posted it here for the community and the AWS team to look into. This is not a VIPS problem and I'll update here if I find anything better.

[baxhul]

@munirhossain I am currently having the same problem. Did you manage to make it work? If not can you please explain your workaround?

[baxhul]

@munirhossain I managed to made it work. Had some fun replacing some dependencies from your package and at some point it worked:

https://drive.google.com/file/d/1dIuyObvLWlTFGDhzFS7EY5ikNNvNfHw1/view

[munirhossain]

Hi, Sorry for late reply. Good to know that you got it worked. If you don't mind please put a note in the git or somewhere how you fixed it. Please send me the link as well.

Best regards, Munir

On Mon, Oct 22, 2018, 1:55 AM baxhul notifications@github.com wrote:

@munirhossain https://github.com/munirhossain I managed to made it work. Had some fun replacing some dependencies from your package and at some point it worked:

https://drive.google.com/file/d/1dIuyObvLWlTFGDhzFS7EY5ikNNvNfHw1/view

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/libvips/pyvips/issues/42#issuecomment-431772063, or mute the thread https://github.com/notifications/unsubscribe-auth/AAzrIqtHyQPEs_qJxRdRDOCFn0OYfvktks5unYf1gaJpZM4UAK3q .

[AntonOellerer]

Hey @baxhul could you maybe report how you solved the issue in more detail? I am facing the same right now.