Zappa failing due to newer version of setuptools

[sridhar562345]

https://github.com/zappa/Zappa/blob/39f75e76d28c1a08d4de6501e6f794fe988cbc98/zappa/core.py#L22

https://github.com/zappa/Zappa/blob/39f75e76d28c1a08d4de6501e6f794fe988cbc98/zappa/core.py#L690

Env details

Python version: 3.10
pip version: 24.2
setuptools version: 75.0.0

Logs

  File "/_venv/lib/python3.10/site-packages/zappa/core.py", line 690, in create_lambda_zip
        copy_tree(temp_package_path, temp_project_path, update=True)
  File "/_venv/lib/python3.10/site-packages/setuptools/_distutils/dir_util.py", line 155, in copy_tree
    return list(itertools.chain.from_iterable(map(copy_one, names)))
  File "/_venv/lib/python3.10/site-packages/setuptools/_distutils/dir_util.py", line 197, in _copy_one
    file_util.copy_file(
  File "/_venv/lib/python3.10/site-packages/setuptools/_distutils/file_util.py", line 104, in copy_file
    from distutils._modified import newer
ModuleNotFoundError: No module named 'distutils._modified'

[sridhar562345]

We already have below function: https://github.com/zappa/Zappa/blob/39f75e76d28c1a08d4de6501e6f794fe988cbc98/zappa/utilities.py#L45

Replacing the distutils usage with copytree in utilities.py should solve the problem.

Let me know if doing so is sufficient or not?

[RicardoNeto]

Following

I temporarilly forced the previous version of setup-tools and it worked

[carlos-schwabe]

To which version of setup-tools did you revert to?

[RicardoNeto]

setuptools==74.1.2

@carlos-schwabe

[shikarkhane]

For anyone else who is using Github actions to deploy and using pipenv, you will have tough time fixing the setuptools version because #pipenv takes the latest.

So you will have update the virtualenv after install like pipenv install -r requirements.txt --python 3.8 pipenv run pip install setuptools==74.1.2

[carlos-schwabe]

@RicardoNeto Even that wouldn't work for me sadly. I had to make a little tweak to the workflow file to make it work properly.

For reference and if anyone else is experiencing issues with GitHub Actions: `- name: Create virtual env with Python3.8 run: | pip install virtualenv python3.8 -m virtualenv env

• name: Install Requirements run: | . env/bin/activate python -m pip install --upgrade pip pip install -r requirements.txt pip install setuptools==69.5.1 deactivate
• name: Install Zappa run: | . env/bin/activate pip install zappa deactivate
  • name: Update application using zappa run: | . env/bin/activate zappa update prod deactivate`

[jturan]

A bit concerning that this PR is still sitting in a review state. Every one of my personal and work-related GitHub Actions deployment scripts using Zappa are failing because of this bug.

[vhiremath]

I started seeing this issue today with my zappa deploy to aws lambda. I have not updated/changed my pyproject.toml or poetry.lock file for last couple of months. Yet this error occurred with latest deploy attempt.

Traceback (most recent call last):
  File "/code/.venv/lib/python3.11/site-packages/zappa/cli.py", line 3048, in handle
    sys.exit(cli.handle())
             ^^^^^^^^^^^^
  File "/code/.venv/lib/python3.11/site-packages/zappa/cli.py", line 521, in handle
    self.dispatch_command(self.command, stage)
  File "/code/.venv/lib/python3.11/site-packages/zappa/cli.py", line 574, in dispatch_command
    self.update(
  File "/code/.venv/lib/python3.11/site-packages/zappa/cli.py", line 980, in update
    self.create_package()
  File "/code/.venv/lib/python3.11/site-packages/zappa/cli.py", line 2453, in create_package
    self.zip_path = self.zappa.create_lambda_zip(  # type: ignore[attr-defined]
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/code/.venv/lib/python3.11/site-packages/zappa/core.py", line 690, in create_lambda_zip
    copy_tree(temp_package_path, temp_project_path, update=True)
==============
  File "/code/.venv/lib/python3.11/site-packages/setuptools/_distutils/dir_util.py", line 155, in copy_tree
    return list(itertools.chain.from_iterable(map(copy_one, names)))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/code/.venv/lib/python3.11/site-packages/setuptools/_distutils/dir_util.py", line 186, in _copy_one
    yield from copy_tree(
               ^^^^^^^^^^
  File "/code/.venv/lib/python3.11/site-packages/setuptools/_distutils/dir_util.py", line 155, in copy_tree
    return list(itertools.chain.from_iterable(map(copy_one, names)))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/code/.venv/lib/python3.11/site-packages/setuptools/_distutils/dir_util.py", line 197, in _copy_one
    file_util.copy_file(
  File "/code/.venv/lib/python3.11/site-packages/setuptools/_distutils/file_util.py", line 104, in copy_file
    from distutils._modified import newer
ModuleNotFoundError: No module named 'distutils._modified'

Whats the remedy here? My poetry.lock says
[package.extras] test = ["pytest (>=6.0.0)", "setuptools (>=65)"]

[ikkshvaku]

iam facing this issue too. when is the fix expected? I already see PR ready for review. Will this be merged soon ? or any other workaround for zappa users plz? thx

[ikkshvaku]

do zappa follow this sequence?

https://github.com/pypa/setuptools/blob/v75.1.0/_distutils_hack/__init__.py#L16-L23

    warnings.warn(
        "Distutils was imported before Setuptools, but importing Setuptools "
        "also replaces the `distutils` module in `sys.modules`. This may lead "
        "to undesirable behaviors or errors. To avoid these issues, avoid "
        "using distutils directly, ensure that setuptools is installed in the "
        "traditional way (e.g. not an editable install), and/or make sure "
        "that setuptools is always imported before distutils."
    )

I see that setuptools TOT still has modified.py at setuptools_distutils Why do we still see the issue in discussion?

[ejjick]

Got it to work with the change proposed in the PR

https://github.com/zappa/Zappa/pull/1352/commits/71d834a60a4af8e62d2795f17093798af66914b0

[ikkshvaku]

My zappa deploy is triggered from Docker CI/CD How can I apply this fix proposed in PR in such case?

[jturan]

For the poetry users, I was able to fix this by pinning an older version of setuptools, similar to the other responses above:

poetry add setuptools@69.5.1

[anthonyhashemi]

Looks like this has been fixed on main with the merging of https://github.com/zappa/Zappa/pull/1352

When can we expect a release including the fix?

[psunny28]

@anthonyhashemi

https://github.com/zappa/Zappa/pull/1355

I was able to solve my issue with this PR

[keeganwhite]

For anyone else who is using Github actions to deploy and using pipenv, you will have tough time fixing the setuptools version because #pipenv takes the latest.

So you will have update the virtualenv after install like pipenv install -r requirements.txt --python 3.8 pipenv run pip install setuptools==74.1.2

this worked for me. THANK YOU SO MUCH!