zappa package fails with zappa from git and slim_handler

jneves commented 6 years ago

Context

If you're trying to run a zappa as an editable from a git repo, it works, except if you have slim_handler enabled.

Expected Behavior

It should build a zip with zappa inside.

Actual Behavior

It fails with a traceback like this when building the handler_venv:

Traceback (most recent call last):
  File "/Users/joao/testes/zappa-1/1/ve/src/zappa/zappa/cli.py", line 2693, in handle
    sys.exit(cli.handle())
  File "/Users/joao/testes/zappa-1/1/ve/src/zappa/zappa/cli.py", line 504, in handle
    self.dispatch_command(self.command, stage)
  File "/Users/joao/testes/zappa-1/1/ve/src/zappa/zappa/cli.py", line 543, in dispatch_command
    self.package(self.vargs['output'])
  File "/Users/joao/testes/zappa-1/1/ve/src/zappa/zappa/cli.py", line 633, in package
    self.create_package(output)
  File "/Users/joao/testes/zappa-1/1/ve/src/zappa/zappa/cli.py", line 2171, in create_package
    venv=self.zappa.create_handler_venv(),
  File "/Users/joao/testes/zappa-1/1/ve/src/zappa/zappa/core.py", line 403, in create_handler_venv
    copytree(os.path.join(current_site_packages_dir, z), os.path.join(venv_site_packages_dir, z))
  File "/Users/joao/testes/zappa-1/1/ve/src/zappa/zappa/utilities.py", line 37, in copytree
    lst = os.listdir(src)
NotADirectoryError: [Errno 20] Not a directory: '/Users/joao/testes/zappa-1/1/ve/lib/python3.6/site-packages/zappa.egg-link

Possible Fix

Use the function for copying editables as when slim_handler is false.

Steps to Reproduce

With the following zappa_settings.json:

{
    "dev": {
        "project_name": "1",
        "runtime": "python3.6",
    "slim_handler": true
    }
}

Run on a virtualenv: pip install -e git+https://github.com/miserlou/zappa#egg=zappa and then zappa package.

Your Environment

Zappa version used: 0.46.1 from git
Operating System and Python version: Mac OS X 10.12.6, python 3.6.4 (from home brew)

The output of pip freeze:

argcomplete==1.9.3
base58==1.0.0
boto3==1.7.45
botocore==1.10.45
certifi==2018.4.16
cfn-flip==1.0.3
chardet==3.0.4
click==6.7
docutils==0.14
durationpy==0.5
first==2.0.1
future==0.16.0
hjson==3.0.1
idna==2.7
jmespath==0.9.3
kappa==0.6.0
lambda-packages==0.20.0
pip-tools==2.0.2
placebo==0.8.1
python-dateutil==2.6.1
python-slugify==1.2.4
PyYAML==3.12
requests==2.19.1
s3transfer==0.1.13
six==1.11.0
toml==0.9.4
tqdm==4.19.1
troposphere==2.3.0
Unidecode==1.0.22
urllib3==1.23
Werkzeug==0.14.1
wsgi-request-logger==0.4.6
-e git+https://github.com/miserlou/zappa@d99c193e32733946fb52a4f9b2bdfd1d2929ba49#egg=zappa

Your zappa_settings.py: already above.

coltyharrison commented 6 years ago

I was able to solve this by cloning the repo, navigating to it in my virtual environment and using python setup.py install. This installs it in a readable format.

lonsdale8734 commented 5 years ago

There is another solution. Just remove the "-e" flag in the last line of the requirements.txt file and then reinstall the zappa package. If you have already installed zappa as editable package, you'll remove the editable package first.

chrisbolman commented 5 years ago

having this same issue with 0.47.1 (python 3.6.5), with or without slim_handler

"dev": {
        "app_function": "app.init.app",
        "profile_name": "...",
        "manage_roles": false,
        "role_name":"...",
        "role_arn": "...",
        "project_name": "...",
        "runtime": "python3.6",
        "s3_bucket": "..."
    }

Traceback (most recent call last):
  File "/Users/chrisbolman/.pyenv/versions/3.6.5/lib/python3.6/site-packages/zappa/cli.py", line 2712, in handle
    sys.exit(cli.handle())
  File "/Users/chrisbolman/.pyenv/versions/3.6.5/lib/python3.6/site-packages/zappa/cli.py", line 509, in handle
    self.dispatch_command(self.command, stage)
  File "/Users/chrisbolman/.pyenv/versions/3.6.5/lib/python3.6/site-packages/zappa/cli.py", line 546, in dispatch_command
    self.deploy(self.vargs['zip'])
  File "/Users/chrisbolman/.pyenv/versions/3.6.5/lib/python3.6/site-packages/zappa/cli.py", line 717, in deploy
    self.create_package()
  File "/Users/chrisbolman/.pyenv/versions/3.6.5/lib/python3.6/site-packages/zappa/cli.py", line 2227, in create_package
    disable_progress=self.disable_progress
  File "/Users/chrisbolman/.pyenv/versions/3.6.5/lib/python3.6/site-packages/zappa/core.py", line 603, in create_lambda_zip
    copytree(site_packages, temp_package_path, metadata=False, symlinks=False, ignore=shutil.ignore_patterns(*excludes))
  File "/Users/chrisbolman/.pyenv/versions/3.6.5/lib/python3.6/site-packages/zappa/utilities.py", line 41, in copytree
    lst = os.listdir(src)
FileNotFoundError: [Errno 2] No such file or directory: '/Users/chrisbolman/Dropbox/br/venv-slack/lib/python3.6/site-packages'

sa77 commented 5 years ago

@ChrisBolman were you able to get it working on python 3.6 ? I am having a similar problem as yours. I've tried downgrading Zappa to 0.45.1 but didn't solve it.

chrisbolman commented 4 years ago

@sa77 unfortunately no, by default, even upgrading to zappa 0.51.0 by default I still get PATH issue between .pyenv/versions/3.6.5/lib/python3.6/site-packages/zappa/utilities.py and /Users/.../lib/python3.6/site-packages (outside pyenv). reinstalling python 3.6.5, creating a new virtualenv, and/or uninstalling + reinstalling zappa don't automatically resolve.

Miserlou / Zappa