Zappa not working with spacy

[niladri-paul]

My API was working perfectly fine with AWS lambda using Zappa. But I need spacy for my code to run. The moment I install spacy==2.0.13 and update the zappa dev , the api stops working. It gives 502 error Following is returned as the response body:

{'message': 'An uncaught exception happened while servicing this request. You can investigate this with the zappa tail command.', 'traceback': ['Traceback (most recent call last):\\n', ' File \"/var/task/handler.py\", line 531, in handler\\n with Response.from_app(self.wsgi_app, environ) as response:\\n', ' File \"/var/task/werkzeug/wrappers/base_response.py\", line 287, in from_app\\n return cls(*_run_wsgi_app(app, environ, buffered))\\n', ' File \"/var/task/werkzeug/wrappers/base_response.py\", line 26, in _run_wsgi_app\\n return _run_wsgi_app(*args)\\n', ' File \"/var/task/werkzeug/test.py\", line 1119, in run_wsgi_app\\n app_rv = app(environ, start_response)\\n', \"TypeError: 'NoneType' object is not callable\\n\"]}

Please help me resolving this issue.

Log from Zappa tail :

File "/var/task/handler.py", line 602, in lambda_handler
 return LambdaHandler.lambda_handler(event, context)
 File "/var/task/handler.py", line 245, in lambda_handler
 handler = cls()
 File "/var/task/handler.py", line 102, in __init__
 self.load_remote_project_archive(project_archive_path)
 File "/var/task/handler.py", line 174, in load_remote_project_archive
 t.extractall(project_folder)
 File "/var/lang/lib/python3.6/tarfile.py", line 2010, in extractall
 numeric_owner=numeric_owner)
 File "/var/lang/lib/python3.6/tarfile.py", line 2052, in extract
 numeric_owner=numeric_owner)
 File "/var/lang/lib/python3.6/tarfile.py", line 2122, in _extract_member
 self.makefile(tarinfo, targetpath)
 File "/var/lang/lib/python3.6/tarfile.py", line 2171, in makefile
 copyfileobj(source, target, tarinfo.size, ReadError, bufsize)
 File "/var/lang/lib/python3.6/tarfile.py", line 252, in copyfileobj
 dst.write(buf)

Log from Cloudwatch Console:

[Errno 28] No space left on device: OSError
Traceback (most recent call last):
 File "/var/task/handler.py", line 602, in lambda_handler
   return LambdaHandler.lambda_handler(event, context)
 File "/var/task/handler.py", line 245, in lambda_handler
   handler = cls()
 File "/var/task/handler.py", line 102, in __init__
   self.load_remote_project_archive(project_archive_path)
 File "/var/task/handler.py", line 174, in load_remote_project_archive
   t.extractall(project_folder)
 File "/var/lang/lib/python3.6/tarfile.py", line 2010, in extractall
   numeric_owner=numeric_owner)
 File "/var/lang/lib/python3.6/tarfile.py", line 2052, in extract
   numeric_owner=numeric_owner)
 File "/var/lang/lib/python3.6/tarfile.py", line 2122, in _extract_member
   self.makefile(tarinfo, targetpath)
 File "/var/lang/lib/python3.6/tarfile.py", line 2171, in makefile
   copyfileobj(source, target, tarinfo.size, ReadError, bufsize)
 File "/var/lang/lib/python3.6/tarfile.py", line 252, in copyfileobj
   dst.write(buf)
OSError: [Errno 28] No space left on device

Output of pip3 freeze:

antlr4-python3-runtime==4.7.1
argcomplete==1.9.3
boto3==1.10.14
botocore==1.13.14
certifi==2019.9.11
cfn-flip==1.2.2
chardet==3.0.4
Click==7.0
cymem==1.31.2
cytoolz==0.9.0.1
decorator==4.4.1
dill==0.2.9
docutils==0.15.2
durationpy==0.5
Flask==1.1.1
future==0.16.0
hjson==3.0.1
idna==2.8
itsdangerous==1.1.0
Jinja2==2.10.3
jmespath==0.9.3
kappa==0.6.0
lambda-packages==0.20.0
MarkupSafe==1.1.1
mpmath==1.1.0
msgpack==0.6.2
msgpack-numpy==0.4.4
murmurhash==0.28.0
networkx==2.2
nltk==3.4
numpy==1.17.0
pandas==0.24.2
Pint==0.8.1
pkg-resources==0.0.0
plac==0.9.6
placebo==0.9.0
preshed==1.0.1
pylatexenc==1.5
python-dateutil==2.6.1
python-slugify==1.2.4
pytz==2019.3
PyYAML==5.1.2
regex==2018.1.10
requests==2.22.0
s3transfer==0.2.1
scipy==1.1.0
singledispatch==3.4.0.3
six==1.13.0
spacy==2.0.13
sympy==1.2
thinc==6.10.3
toml==0.10.0
toolz==0.10.0
tqdm==4.19.1
troposphere==2.5.2
ujson==1.35
Unidecode==1.1.1
urllib3==1.25.6
Werkzeug==0.16.0
wrapt==1.10.11
wsgi-request-logger==0.4.6
zappa==0.48.2

Zappa Settngs.json:

{
    "dev": {
        "app_function": "app.app",
        "aws_region": "region_name",
        "profile_name": "default",
        "project_name": "project_name",
        "runtime": "python3.6",
        "s3_bucket": "bucket_name",
         "manage_roles": false, 
         "role_name": "My_role_name", 
        "role_arn": "Role_arn",
         "slim_handler": true,
     "api_key_required": true,
     "memory_size": 3008,
    }
}

[moviedatascience]

Did you ever get this working? I'm running into the same issue

[niladri-paul]

I upgraded the version of spacy. Spacy 2.2.0 and onwards work fine with AWS.

Thanks, Niladri

On Sun, Feb 2, 2020 at 2:03 PM Eric notifications@github.com wrote:

Did you ever get this working? I'm running into the same issue

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/Miserlou/Zappa/issues/1957?email_source=notifications&email_token=AJOVX43JY7NTROPVR24Y7FDRA2ANPA5CNFSM4JLDIDLKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKRQ7AQ#issuecomment-581111682, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJOVX42EEY4HYCO76M745SLRA2ANPANCNFSM4JLDIDLA .

[moviedatascience]

Oh I actually realized this project version isn't using Spacy, but Gensim. Will have to mess around with Gensim version I guess, as it's currently on the latest.

[jneves]

The error message points out to the disk space running out. In lambda there's a 500MB hard limit in total. You seem to have hit that.