search

Miserlou / Zappa

Serverless Python
https://blog.zappa.io/
MIT License
11.88k stars 1.2k forks source link

Cytoolz module doesn't work with zappa deployment #1717

Open mberledgylabs opened 6 years ago

mberledgylabs commented 6 years ago

Context

I am trying to deploy flask-app to AWS Lambda with zappa. The app works fine in active environment and is deployed without errors to AWS Lambda. But the status check doesn't work because of an import error in cytoolz. I checked that itertoolz.pyx is present in the cytoolz folder in the package made by zappa. I also tried to resolve an issue by adding cython to the environment but it didn't help.

The full stacktrace is below:

No module named 'cytoolz.itertoolz': ModuleNotFoundError Traceback (most recent call last): File "/var/task/handler.py", line 580, in lambda_handler return LambdaHandler.lambda_handler(event, context) File "/var/task/handler.py", line 245, in lambda_handler handler = cls() File "/var/task/handler.py", line 139, in init self.app_module = importlib.import_module(self.settings.APP_MODULE) File "/var/lang/lib/python3.6/importlib/init.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "", line 978, in _gcd_import File "", line 961, in _find_and_load File "", line 950, in _find_and_load_unlocked File "", line 655, in _load_unlocked File "", line 678, in exec_module File "", line 205, in _call_with_frames_removed File "/var/task/src/app.py", line 2, in import cytoolz File "/var/task/cytoolz/init.py", line 1, in from .itertoolz import * ModuleNotFoundError: No module named 'cytoolz.itertoolz'

As the code works locally I believe the problem has to do with how zappa deploys to lambda.

Expected Behavior

There should not be an error when importing cytoolz.

Actual Behavior

The app is deployed but doesn't work because of the import error.

Steps to Reproduce

  1. The code of app.py which must be enough to reproduce the problem:

`from flask import Flask import cytoolz

def create_app(): flask_app = Flask(name) return flask_app

app = create_app()

@app.route("/") def index(): return "Hello world!"`

Your Environment

mberledgylabs commented 5 years ago

I have managed to make it work by using Zappa with Docker . Although, I don't know whether it should be considered a bug.

qria commented 5 years ago

I had the same problem with same error message and this seems to be the problem: https://github.com/ethereum/web3.py/issues/554#issuecomment-358077654

It says currently cytoolz is built with CPython option incompatible with AWS's version and recommends that you build cytoolz locally. However I was not able to find a method to bundle locally built library with zappa.

So I think this is a bug with either cytoolz or aws and current best method to circumvent it is to use zappa with docker like you said. Thanks for the heads up it really helped.

phani9 commented 3 years ago

@qria @mberledgylabs

do you have any instructions for using Zappa with docker ? My local execution works fine, but when I run from AWS Lambda, I'm getting below error.

Unable to import module: No module named 'cytoolz.itertoolz'

These are the two related links I found.

  1. https://github.com/Miserlou/zappa-blog/blob/master/posts/simplified-aws-lambda-deployments-with-docker-and-zappa.md
  2. https://github.com/Miserlou/zappa-blog/blob/master/posts/docker-zappa-and-python3.md
pfcodes commented 3 years ago

@qria @mberledgylabs

do you have any instructions for using Zappa with docker ? My local execution works fine, but when I run from AWS Lambda, I'm getting below error.

Unable to import module: No module named 'cytoolz.itertoolz'

Did you figure this out?

0xRadix commented 3 years ago

@qria @mberledgylabs @phani9 @pfcodes

Were you able to figure this out? Running into the same issue.

pfcodes commented 3 years ago

@qria @mberledgylabs @phani9 @pfcodes

Were you able to figure this out? Running into the same issue.

@rdxpt Yes. You have to build everything in the same environment the lambda runs in. You can use docker to do this. I used this guide. It's annoying but you have to remember to build within the docker container every time. I'm eventually going to set up a CI/CD environment so I'm not manually doing this.