[Migrated] Tensorflow and Zappa: No module named 'tensorflow' ModuleNotFoundError

Originally from: https://github.com/Miserlou/Zappa/issues/896 by azagovora

I am exploring the possibility of migration our prediction services to Lambda. I am configuring a simple Flask application which depends on tensorflow to run with Zappa and Python 3.6. Unfortunately, I got the error below. The application works with Python 2.7 without any errors.

Actual Behavior

Call fails with internal server errors

Steps to Reproduce

app.py

import sys

from flask import Flask
import tensorflow as tf

app = Flask(__name__)
app.debug = True

print(sys.maxunicode)

@app.route('/')
def index():
    x = tf.ones(5) * 2
    with tf.Session():
        y = x.eval()
    return "Hello, world!" + str(y), 200

# We only need this for local development.
if __name__ == '__main__':
    app.run()

zappa_settings.json

{
    "dev": {
        "app_function": "app.app",
        "profile_name": "default",
        "s3_bucket": "zappa-tensorflow",
        "runtime": "python3.6",
        "slim_handler": true,
        "debug" : true
    }
}

Create the 2 files listed above 1 Set up virtualenv 2 zappa deploy dev

After the application deploys it will respond to all requests with internal sever errors for any requests

Taking a look at what is going on zappa tail dev

1495821395423] Instancing..
[1495821395424] Failed to find library...right filename?
[1495821396121] No module named 'tensorflow': ModuleNotFoundError
Traceback (most recent call last):
  File "/var/task/handler.py", line 484, in lambda_handler
  return LambdaHandler.lambda_handler(event, context)
  File "/var/task/handler.py", line 240, in lambda_handler
  handler = cls()
  File "/var/task/handler.py", line 129, in __init__
  self.app_module = importlib.import_module(self.settings.APP_MODULE)
  File "/var/lang/lib/python3.6/importlib/__init__.py", line 126, in import_module
  return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 978, in _gcd_import
  File "<frozen importlib._bootstrap>", line 961, in _find_and_load
  File "<frozen importlib._bootstrap>", line 950, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 655, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 205, in _call_with_frames_removed
  File "/tmp/lambda-tensorfl/app.py", line 7, in <module>
  import tensorflow as tf
ModuleNotFoundError: No module named 'tensorflow'
[1495821396121] Traceback (most recent call last):
[1495821396121] File "/var/runtime/awslambda/bootstrap.py", line 226, in handle_event_request
[1495821396121] result = request_handler(json_input, context)
[1495821396121] File "/var/task/handler.py", line 484, in lambda_handler
[1495821396121] return LambdaHandler.lambda_handler(event, context)
[1495821396121] File "/var/task/handler.py", line 240, in lambda_handler
[1495821396121] handler = cls()
[1495821396121] File "/var/task/handler.py", line 129, in __init__
[1495821396122] self.app_module = importlib.import_module(self.settings.APP_MODULE)
[1495821396122] File "/var/lang/lib/python3.6/importlib/__init__.py", line 126, in import_module
[1495821396122] return _bootstrap._gcd_import(name[level:], package, level)
[1495821396122] File "<frozen importlib._bootstrap>", line 978, in _gcd_import
[1495821396122] File "<frozen importlib._bootstrap>", line 961, in _find_and_load
[1495821396122] File "<frozen importlib._bootstrap>", line 950, in _find_and_load_unlocked
[1495821396122] File "<frozen importlib._bootstrap>", line 655, in _load_unlocked
[1495821396122] File "<frozen importlib._bootstrap_external>", line 678, in exec_module
[1495821396122] File "<frozen importlib._bootstrap>", line 205, in _call_with_frames_removed
[1495821396122] File "/tmp/lambda-tensorfl/app.py", line 7, in <module>
[1495821396122] import tensorflow as tf
[1495821396122] ModuleNotFoundError: No module named 'tensorflow'
[1495821396122] During handling of the above exception, another exception occurred:
[1495821396122] Traceback (most recent call last):
[1495821396122] File "/var/runtime/awslambda/bootstrap.py", line 463, in <module>
[1495821396122] main()
[1495821396122] File "/var/runtime/awslambda/bootstrap.py", line 459, in main
[1495821396123] handle_event_request(request_handler, invokeid, event_body, context_objs, invoked_function_arn)
[1495821396123] File "/var/runtime/awslambda/bootstrap.py", line 240, in handle_event_request
[1495821396123] result = to_json(result)
[1495821396123] File "/var/runtime/awslambda/bootstrap.py", line 215, in to_json
[1495821396123] return json.dumps(obj, default=decimal_serializer)
[1495821396123] File "/var/lang/lib/python3.6/json/__init__.py", line 238, in dumps
[1495821396123] **kw).encode(obj)
[1495821396123] File "/var/lang/lib/python3.6/json/encoder.py", line 199, in encode
[1495821396123] chunks = self.iterencode(o, _one_shot=True)
[1495821396123] File "/var/lang/lib/python3.6/json/encoder.py", line 257, in iterencode
[1495821396123] return _iterencode(o, 0)
[1495821396123] File "/var/runtime/awslambda/bootstrap.py", line 104, in decimal_serializer
[1495821396123] raise TypeError(repr(o) + " is not JSON serializable")
[1495821396123] TypeError: <FrameSummary file /var/task/handler.py, line 484 in lambda_handler> is not JSON serializable
[1495821396253] No module named 'tensorflow'
ModuleNotFoundError

Your Environment

appdirs==1.4.3 argcomplete==1.8.2 base58==0.2.4 boto3==1.4.4 botocore==1.5.40 click==6.7 docutils==0.13.1 durationpy==0.4 Flask==0.12.2 future==0.16.0 futures==3.1.1 hjson==2.0.2 itsdangerous==0.24 Jinja2==2.9.6 jmespath==0.9.2 kappa==0.6.0 lambda-packages==0.15.0 MarkupSafe==1.0 numpy==1.12.1 packaging==16.8 placebo==0.8.1 protobuf==3.3.0 pyparsing==2.2.0 python-dateutil==2.6.0 python-slugify==1.2.4 PyYAML==3.12 requests==2.14.2 s3transfer==0.1.10 six==1.10.0 tensorflow==1.1.0 toml==0.9.2 tqdm==4.11.2 troposphere==1.9.3 Unidecode==0.4.20 Werkzeug==0.12.2 wsgi-request-logger==0.4.6 zappa==0.42.0

Zappa version used: 42.0
Operating System and Python version: macOS Sierra 10.12.5, Python 3.6.1

zappa / Zappa