cannot import name _imaging: ImportError

johey commented 7 years ago

Trying to follow Rich Jones - "Building Serverless Microservices with Zappa and Flask" tutorial. It works perfectly fine until I add dependency towards Pillow. Locally it continues working fine, using Flask webserver, but deployed on AWS using Zappa, it failes on ImportError: cannot import name _imaging.

Expected Behavior

Function should work, presenting me with a web page.

Actual Behavior

I get a json error in the web browser:

"{u'message': u'An uncaught exception happened while servicing this request. You can investigate this with thezappa tailcommand.', u'traceback': ['Traceback (most recent call last):\\n', ' File \"/var/task/handler.py\", line 427, in handler\\n response = Response.from_app(self.wsgi_app, environ)\\n', ' File \"c:\\\\users\\\\johan\\\\appdata\\\\local\\\\temp\\\\pip-build-s_0wot\\\\Werkzeug\\\\werkzeug\\\\wrappers.py\", line 903, in from_app\\n', ' File \"c:\\\\users\\\\johan\\\\appdata\\\\local\\\\temp\\\\pip-build-s_0wot\\\\Werkzeug\\\\werkzeug\\\\test.py\", line 884, in run_wsgi_app\\n', \"TypeError: 'NoneType' object is not callable\\n\"]}"

Output from zappa tail:

(Werkzeug 0.12.2 (c:\users\johan\envs\smolinscy-p27\lib\site-packages), Requirement.parse('Werkzeug==0.12'), set([u'zappa']))
Calling tail for stage dev..
[1497208530396] cannot import name _imaging: ImportError
Traceback (most recent call last):
  File "/var/task/handler.py", line 484, in lambda_handler
  return LambdaHandler.lambda_handler(event, context)
  File "/var/task/handler.py", line 240, in lambda_handler
  handler = cls()
  File "/var/task/handler.py", line 129, in __init__
  self.app_module = importlib.import_module(self.settings.APP_MODULE)
  File "/usr/lib64/python2.7/importlib/__init__.py", line 37, in import_module
  __import__(name)
  File "/var/task/my_app.py", line 2, in <module>
  from PIL import Image
  File "c:\users\johan\appdata\local\temp\pip-build-s_0wot\Pillow\PIL\Image.py", line 56, in <module>
ImportError: cannot import name _imaging
[1497208532972] [DEBUG] 2017-06-11T19:15:32.971Z 571d3c1f-4eda-11e7-b357-01a1868f9106 Zappa Event: {u'body': None, u'resource': u'/', u'requestContext': {u'resourceId': u'4kzi4znk7g', u'apiId': u'0tetn48w41', u'resourcePath': u'/', u'httpMethod': u'GET', u'requestId': u'570cc107-4eda-11e7-a1c0-7b1585d6d9cf', u'path': u'/dev', u'accountId': u'776316233763', u'identity': {u'apiKey': u'', u'userArn': None, u'cognitoAuthenticationType': None, u'accessKey': None, u'caller': None, u'userAgent': u'python-requests/2.17.3', u'user': None, u'cognitoIdentityPoolId': None, u'cognitoIdentityId': None, u'cognitoAuthenticationProvider': None, u'sourceIp': u'178.174.128.228', u'accountId': None}, u'stage': u'dev'}, u'queryStringParameters': None, u'httpMethod': u'GET', u'pathParameters': None, u'headers': {u'Via': u'1.1 6cde3c778df412041adc7610331b57bc.cloudfront.net (CloudFront)', u'Accept-Encoding': u'gzip, deflate', u'CloudFront-Is-SmartTV-Viewer': u'false', u'CloudFront-Forwarded-Proto': u'https', u'X-Forwarded-For': u'178.174.128.228, 205.251.218.87', u'CloudFront-Viewer-Country': u'SE', u'Accept': u'*/*', u'User-Agent': u'python-requests/2.17.3', u'X-Amzn-Trace-Id': u'Root=1-593d96d4-2fec7f74248984475b8e424f', u'Host': u'0tetn48w41.execute-api.eu-west-1.amazonaws.com', u'X-Forwarded-Proto': u'https', u'X-Amz-Cf-Id': u'29tZbyGeMqezMQ3WHLUKWOCEX7LQRO4E5TROmNPJaM7MUw5DmLkMIw==', u'CloudFront-Is-Tablet-Viewer': u'false', u'X-Forwarded-Port': u'443', u'CloudFront-Is-Mobile-Viewer': u'false', u'CloudFront-Is-Desktop-Viewer': u'true'}, u'stageVariables': None, u'path': u'/', u'isBase64Encoded': False}
[1497208532972] 'NoneType' object is not callable

Steps to Reproduce

git clone git@github.com:Miserlou/serverless-imagehost.git
virtualenv ... activate ...
pip install flask zappa boto3 Pillow
set up s3 bucket and point in zappa_settings.json
zappa deploy dev
browse to the url given by zappa and cry. :(

Your Environment

Zappa version used: 0.42.0
Operating System and Python version: Windows 10, Python 2.7.13

The output of pip freeze:

argcomplete==1.8.2
base58==0.2.4
boto3==1.4.4
botocore==1.5.40
certifi==2017.4.17
chardet==3.0.4
click==6.7
docutils==0.13.1
durationpy==0.4
Flask==0.12.2
future==0.16.0
futures==3.1.1
hjson==2.0.2
idna==2.5
itsdangerous==0.24
Jinja2==2.9.6
jmespath==0.9.2
kappa==0.6.0
lambda-packages==0.15.0
MarkupSafe==1.0
olefile==0.44
Pillow==4.1.1
placebo==0.8.1
python-dateutil==2.6.0
python-slugify==1.2.4
PyYAML==3.12
requests==2.17.3
s3transfer==0.1.10
six==1.10.0
toml==0.9.2
tqdm==4.11.2
troposphere==1.9.4
Unidecode==0.4.20
urllib3==1.21.1
Werkzeug==0.12.2
wsgi-request-logger==0.4.6
zappa==0.42.0

Your zappa_settings.py:

{
"dev": {
   "s3_bucket": "smolinscy",
   "app_function": "my_app.app"
}
}

jonatasbaldin commented 7 years ago

Hi!

Just tested this here and it worked fine for me, using the same Python version and requirements file., although I'm using OSX.

Did you put the same bucket named used in zappa_settings.json in the BUCKET_NAME variable inside my_app.py?

johey commented 7 years ago

Ok, thanks for testing it out! That is strange indeed. Yes, I am consistent regarding bucket name. Will spend some more time trying to understand what is causing the problem.

jonatasbaldin commented 7 years ago

I'll try to get a Windows machine to test it out too. If you have any updates, please let us know!

johey commented 7 years ago

Just tried on a GNU/Linux machine. Works perfectly! Now I'm happy. Who needs Windows anyway. :) Yet, I'll leave the issue open, as there actually seems to be a problem.

jonatasbaldin commented 7 years ago

Thats awesome @johey! Let's wais if any contributor with a Windows machine could test it out!

MattSegal commented 6 years ago

I think I've got a work-around for Windows (using Python 2.7):

The fix is to try really hard to make sure that you upload the precompiled lambda package version of Pillow instead of your local Windows-friendly version of Pillow:

Ensure you are using pillow==3.4.2 - the same version as here
make a dummy folder at ./{venv-name}/Lib/site-packages/pillow so that zappa.core.Zappa.get_installed_packages knows that you are using pillow
Run zappa update and you should see pillow==3.4.2: Using precompiled lambda package (which implies success)

You can get a feel for what's going on by reading zappa.core.Zappa.get_installed_packages and zappa.core.Zappa.create_lambda_zip and then playing with the following code:

import os
import pip
import subprocess
from pprint import pprint as p

from lambda_packages import lambda_packages as lambda_packages_orig
from zappa.core import Zappa

lambda_packages = {package_name.lower(): val for package_name, val in lambda_packages_orig.items()}
p(lambda_packages['pillow'])

z = Zappa()
venv = z.get_current_venv()
site_packages = os.path.join(venv, 'Lib', 'site-packages')

print "\nPACKAGES IN SITE PACKAGES\n"
p(os.listdir(site_packages))

print "\nPIP INSTALLED PACKAGES\n"
p({
    package.project_name.lower(): package.version
    for package in pip.get_installed_distributions()
})

print "\nFINAL INSTALLED PACKAGES\n"
installed_packages = z.get_installed_packages(site_packages, '')
p(installed_packages)

rquintino commented 5 years ago

just leaving the note I was getting similar error but with flask:

[1554818782296] cannot import name 'Flask': ImportError

zappa initial touch after deploy also failed with http 502 due to this

as this thread suggested, I was using Windows and slim option, tried running same code on docker python image (ubuntu), problem solved! thanks! :)

Miserlou / Zappa