search

Miserlou / Zappa

Serverless Python
https://blog.zappa.io/
MIT License
11.89k stars 1.2k forks source link

Precompiled wheels are not extracted according to PEP491 which leads to import errors. #2104

Open kairichard opened 4 years ago

kairichard commented 4 years ago

When installing manylinux wheels with use_precompiled_packages set to true zappa only unzips the wheels ignoring purelib and platlib folders as specified in PEP491

Expected Behavior

A correctly extracted wheel in the resulting lambda zip:

./tensorflow
./tensorflow/tools
./tensorflow/core
./tensorflow/python
./tensorflow/lite
./tensorflow/libtensorflow_framework.so.1
./tensorflow/include
./tensorflow/_api
./tensorflow/__init__.py
./tensorflow/contrib
./tensorflow/examples
./tensorflow/compiler
./tensorflow-1.14.0.dist-info
./tensorflow-1.14.0.dist-info/RECORD
./tensorflow-1.14.0.dist-info/metadata.json
./tensorflow-1.14.0.dist-info/WHEEL
./tensorflow-1.14.0.dist-info/entry_points.txt
./tensorflow-1.14.0.dist-info/DESCRIPTION.rst
./tensorflow-1.14.0.dist-info/top_level.txt
./tensorflow-1.14.0.dist-info/METADATA

Actual Behavior

When inspecting the resulting lambda zip you can see the following folders:

./tensorflow-1.14.0.data
./tensorflow-1.14.0.data/purelib
./tensorflow-1.14.0.dist-info
./tensorflow-1.14.0.dist-info/RECORD
./tensorflow-1.14.0.dist-info/metadata.json
./tensorflow-1.14.0.dist-info/WHEEL
./tensorflow-1.14.0.dist-info/entry_points.txt
./tensorflow-1.14.0.dist-info/DESCRIPTION.rst
./tensorflow-1.14.0.dist-info/top_level.txt
./tensorflow-1.14.0.dist-info/METADATA

As a result, we get the following error when running the lambda function

No module named 'tensorflow'

Steps to Reproduce

I only tested it with tensorflow=1.14.0 but it seems like a general problem with all wheels that are purelibs as zappa only extracts them and does no further processing see -> https://github.com/Miserlou/Zappa/blob/master/zappa/core.py#L659

Here is the whl -> tensorflow=1.14.9 from https://pypi.org/project/tensorflow/1.14.0/#files

jneves commented 4 years ago

Was the typo 'tensoflow' an accident?

kairichard commented 4 years ago

@jneves aha oh gosh, yes that is a typo.