search

fabfuel / ecs-deploy

Powerful CLI tool to simplify Amazon ECS deployments, rollbacks & scaling
Other
854 stars 148 forks source link

Cannot use ecs-deploy without installing setuptools #217

Closed jnyrup closed 10 months ago

jnyrup commented 11 months ago

I'm using ecs-deploy in Azure DevOps and this morning our deploy pipeline crashed with an error No module named 'pkg_resources'.

Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.12.0/x64/bin/ecs", line 5, in <module>
    from ecs_deploy.cli import ecs
  File "/opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/ecs_deploy/cli.py", line 12, in <module>
    from ecs_deploy.ecs import DeployAction, ScaleAction, RunAction, EcsClient, DiffAction, \
  File "/opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/ecs_deploy/ecs.py", line 12, in <module>
    from dictdiffer import diff
  File "/opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/dictdiffer/__init__.py", line 15, in <module>
    import pkg_resources
ModuleNotFoundError: No module named 'pkg_resources'

As a workaround we manually install setuptools, but I guess that shouldn't be required to use ecs-deploy?

I'm not well-versed in the python ecosystem, but as far as I can tell, the dependency on pkg_resources comes from Dictdiffer. Its dependency on pkg_resources was removed in https://github.com/inveniosoftware/dictdiffer/pull/139 and released in v0.9.0

I see that ecs-deploy explicitly depends on v0.8.0 which was the newest available at the time of its inclusion in ecs-deploy. https://github.com/fabfuel/ecs-deploy/blob/79812c7cf22f977e73a223d60f34dd8a82900da4/setup.py#L21

Fast-forwarding to today, would it be possible (perhaps even desirable) to bump ecs-deploy's dependency on dictdiffer to 0.9.0?

In case you're interested in what changed with our dependencies, here's the diff in the resolved modules between the last successful run and the first failing one.

- boto3-1.28.59
+ boto3-1.28.62
- botocore-1.31.59
+ botocore-1.31.62
- urllib3-1.26.17
+ urllib3-2.0.6
more log ``` Collecting ecs-deploy Downloading ecs-deploy-1.14.0.tar.gz (33 kB) Installing build dependencies: started Installing build dependencies: finished with status 'done' Getting requirements to build wheel: started Getting requirements to build wheel: finished with status 'done' Preparing metadata (pyproject.toml): started Preparing metadata (pyproject.toml): finished with status 'done' Collecting click<9,>=7.1.2 (from ecs-deploy) Obtaining dependency information for click<9,>=7.1.2 from https://files.pythonhosted.org/packages/00/2e/d53fa4befbf2cfa713304affc7ca780ce4fc1fd8710527771b58311a3229/click-8.1.7-py3-none-any.whl.metadata Downloading click-8.1.7-py3-none-any.whl.metadata (3.0 kB) Collecting click-log==0.3.2 (from ecs-deploy) Downloading click_log-0.3.2-py2.py3-none-any.whl (4.6 kB) Collecting botocore>=1.17.47 (from ecs-deploy) Obtaining dependency information for botocore>=1.17.47 from https://files.pythonhosted.org/packages/a8/3f/74138007b045447eac6141c8144efe8e1c9f377cf56c85edfe1111a22f97/botocore-1.31.62-py3-none-any.whl.metadata Downloading botocore-1.31.62-py3-none-any.whl.metadata (6.1 kB) Collecting boto3>=1.14.47 (from ecs-deploy) Obtaining dependency information for boto3>=1.14.47 from https://files.pythonhosted.org/packages/63/e5/8fc4a69186cb15b0dba9c428da73233c89eb18ee03ce56f6bde205ea2006/boto3-1.28.62-py3-none-any.whl.metadata Downloading boto3-1.28.62-py3-none-any.whl.metadata (6.7 kB) Collecting future (from ecs-deploy) Downloading future-0.18.3.tar.gz (840 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 840.9/840.9 kB 27.0 MB/s eta 0:00:00 Installing build dependencies: started Installing build dependencies: finished with status 'done' Getting requirements to build wheel: started Getting requirements to build wheel: finished with status 'done' Preparing metadata (pyproject.toml): started Preparing metadata (pyproject.toml): finished with status 'done' Collecting requests (from ecs-deploy) Obtaining dependency information for requests from https://files.pythonhosted.org/packages/70/8e/0e2d847013cb52cd35b38c009bb167a1a26b2ce6cd6965bf26b47bc0bf44/requests-2.31.0-py3-none-any.whl.metadata Downloading requests-2.31.0-py3-none-any.whl.metadata (4.6 kB) Collecting dictdiffer==0.8.0 (from ecs-deploy) Downloading dictdiffer-0.8.0-py2.py3-none-any.whl (16 kB) Collecting jmespath<2.0.0,>=0.7.1 (from boto3>=1.14.47->ecs-deploy) Downloading jmespath-1.0.1-py3-none-any.whl (20 kB) Collecting s3transfer<0.8.0,>=0.7.0 (from boto3>=1.14.47->ecs-deploy) Obtaining dependency information for s3transfer<0.8.0,>=0.7.0 from https://files.pythonhosted.org/packages/5a/4b/fec9ce18f8874a96c5061422625ba86c3ee1e6587ccd92ff9f5bf7bd91b2/s3transfer-0.7.0-py3-none-any.whl.metadata Downloading s3transfer-0.7.0-py3-none-any.whl.metadata (1.8 kB) Collecting python-dateutil<3.0.0,>=2.1 (from botocore>=1.17.47->ecs-deploy) Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 247.7/247.7 kB 62.7 MB/s eta 0:00:00 Collecting urllib3<2.1,>=1.25.4 (from botocore>=1.17.47->ecs-deploy) Obtaining dependency information for urllib3<2.1,>=1.25.4 from https://files.pythonhosted.org/packages/26/40/9957270221b6d3e9a3b92fdfba80dd5c9661ff45a664b47edd5d00f707f5/urllib3-2.0.6-py3-none-any.whl.metadata Downloading urllib3-2.0.6-py3-none-any.whl.metadata (6.6 kB) Collecting charset-normalizer<4,>=2 (from requests->ecs-deploy) Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/e1/ee/04bbbe050ac4d4ac92ca3fa4f9ada00ee7d2095a91ccadb9a2065943a1bf/charset_normalizer-3.3.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata Downloading charset_normalizer-3.3.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (32 kB) Collecting idna<4,>=2.5 (from requests->ecs-deploy) Downloading idna-3.4-py3-none-any.whl (61 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.5/61.5 kB 21.6 MB/s eta 0:00:00 Collecting certifi>=2017.4.17 (from requests->ecs-deploy) Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata Downloading certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB) Collecting six>=1.5 (from python-dateutil<3.0.0,>=2.1->botocore>=1.17.47->ecs-deploy) Downloading six-1.16.0-py2.py3-none-any.whl (11 kB) Downloading boto3-1.28.62-py3-none-any.whl (135 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 135.8/135.8 kB 40.0 MB/s eta 0:00:00 Downloading botocore-1.31.62-py3-none-any.whl (11.2 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.2/11.2 MB 117.1 MB/s eta 0:00:00 Downloading click-8.1.7-py3-none-any.whl (97 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 97.9/97.9 kB 34.1 MB/s eta 0:00:00 Downloading requests-2.31.0-py3-none-any.whl (62 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.6/62.6 kB 25.8 MB/s eta 0:00:00 Downloading certifi-2023.7.22-py3-none-any.whl (158 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 158.3/158.3 kB 40.9 MB/s eta 0:00:00 Downloading charset_normalizer-3.3.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 138.7/138.7 kB 48.9 MB/s eta 0:00:00 Downloading s3transfer-0.7.0-py3-none-any.whl (79 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 79.8/79.8 kB 29.9 MB/s eta 0:00:00 Downloading urllib3-2.0.6-py3-none-any.whl (123 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 123.8/123.8 kB 46.4 MB/s eta 0:00:00 Building wheels for collected packages: ecs-deploy, future Building wheel for ecs-deploy (pyproject.toml): started Building wheel for ecs-deploy (pyproject.toml): finished with status 'done' Created wheel for ecs-deploy: filename=ecs_deploy-1.14.0-py2.py3-none-any.whl size=27650 sha256=82362d4fd07a7b4b548504aed2c2de9792896343e3e6011c80fb94b243815586 Stored in directory: /home/vsts/.cache/pip/wheels/52/4b/ff/b92eb003aa3c22df5a2b66a73f10baef4dfe4d8f2048f2e11a Building wheel for future (pyproject.toml): started Building wheel for future (pyproject.toml): finished with status 'done' Created wheel for future: filename=future-0.18.3-py3-none-any.whl size=492024 sha256=a005d59d745a91709c82d3dd43511e2d551a6da5eb2af0d0444afddf8ae7ab75 Stored in directory: /home/vsts/.cache/pip/wheels/f1/54/27/4e5cb4bd05116c097877e85a206a2e08d301aaa9b7af719909 Successfully built ecs-deploy future Installing collected packages: dictdiffer, urllib3, six, jmespath, idna, future, click, charset-normalizer, certifi, requests, python-dateutil, click-log, botocore, s3transfer, boto3, ecs-deploy Successfully installed boto3-1.28.62 botocore-1.31.62 certifi-2023.7.22 charset-normalizer-3.3.0 click-8.1.7 click-log-0.3.2 dictdiffer-0.8.0 ecs-deploy-1.14.0 future-0.18.3 idna-3.4 jmespath-1.0.1 python-dateutil-2.8.2 requests-2.31.0 s3transfer-0.7.0 six-1.16.0 urllib3-2.0.6 Traceback (most recent call last): File "/opt/hostedtoolcache/Python/3.12.0/x64/bin/ecs", line 5, in from ecs_deploy.cli import ecs File "/opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/ecs_deploy/cli.py", line 12, in from ecs_deploy.ecs import DeployAction, ScaleAction, RunAction, EcsClient, DiffAction, \ File "/opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/ecs_deploy/ecs.py", line 12, in from dictdiffer import diff File "/opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/dictdiffer/__init__.py", line 15, in import pkg_resources ModuleNotFoundError: No module named 'pkg_resources' ```
fabfuel commented 11 months ago

Hi Jonas,

thanks for reporting this. It seems to be related to Python 3.12, I will look into it!

Best Fabian