search

aws / aws-elastic-beanstalk-cli-setup

Simplified EB CLI installation mechanism.
Apache License 2.0
515 stars 226 forks source link

bundled_installer takes more than one hour #104

Open momotaro98 opened 3 years ago

momotaro98 commented 3 years ago

1. Please specify the following:

2. Description

I use bundled_installer in GitLab CI Runner (Ubuntu) with python:3.9 Image and here's the script I use.

- apt-get update
- apt-get install -y git build-essential zlib1g-dev libssl-dev libncurses-dev libffi-dev libsqlite3-dev libreadline-dev libbz2-dev
- pip install --upgrade pip
- git clone https://github.com/aws/aws-elastic-beanstalk-cli-setup.git
- ./aws-elastic-beanstalk-cli-setup/scripts/bundled_installer

The script had worked within 10 minutes before but since today, the bundled_installer has got timeout because it takes more than one hour.

I think over one hour is too long time. Can this issue be fixed?

The error message from GitLab CI is here.

$ ./aws-elastic-beanstalk-cli-setup/scripts/bundled_installer

==============================================
I. Installing Python                          
==============================================

*************************************************************
1. Determining whether pyenv is already installed and in PATH
*************************************************************
    - pyenv was not found in PATH.

*********************************************************
2. Determining whether pyenv should be cloned from GitHub
*********************************************************

*********************************************************************************
3. Cloning the pyenv GitHub project located at https://github.com/pyenv/pyenv.git
*********************************************************************************
Cloning into '/root/.pyenv-repository'...
Switched to a new branch 'rel-1.2.9'

*******************************************
4. Temporarily export necessary pyenv paths
*******************************************

****************************************************************************
5. Checking whether Python can be downloaded (through curl, wget, or aria2c)
****************************************************************************

************************************************************
6. Installing Python 3.7.2. This step may take a few minutes
************************************************************
Downloading Python-3.7.2.tar.xz...
-> https://www.python.org/ftp/python/3.7.2/Python-3.7.2.tar.xz
Installing Python-3.7.2...
Installed Python-3.7.2 to /root/.pyenv/versions/3.7.2

    - Python 3.7.2 is installed at /root/.pyenv/versions/3.7.2/bin

******************************************************************
7. Installing virtualenv using /root/.pyenv/versions/3.7.2/bin/pip
******************************************************************
Collecting virtualenv
  Downloading https://files.pythonhosted.org/packages/9f/fb/7423637e48cffbb2e567ca113d2b05068f8b457dde998ab487adf7386c86/virtualenv-20.4.2-py2.py3-none-any.whl (7.2MB)
Collecting filelock<4,>=3.0.0 (from virtualenv)
  Downloading https://files.pythonhosted.org/packages/93/83/71a2ee6158bb9f39a90c0dea1637f81d5eef866e188e1971a1b1ab01a35a/filelock-3.0.12-py3-none-any.whl
Collecting importlib-metadata>=0.12; python_version < "3.8" (from virtualenv)
  Downloading https://files.pythonhosted.org/packages/f3/ed/da40116a204abb5c4dd1d929346d33e0d29cedb2cedd18ea98f0385dcd92/importlib_metadata-3.4.0-py3-none-any.whl
Collecting appdirs<2,>=1.4.3 (from virtualenv)
  Downloading https://files.pythonhosted.org/packages/3b/00/2344469e2084fb287c2e0b57b72910309874c3245463acd6cf5e3db69324/appdirs-1.4.4-py2.py3-none-any.whl
Collecting six<2,>=1.9.0 (from virtualenv)
  Downloading https://files.pythonhosted.org/packages/ee/ff/48bde5c0f013094d729fe4b0316ba2a24774b3ff1c52d924a8a4cb04078a/six-1.15.0-py2.py3-none-any.whl
Collecting distlib<1,>=0.3.1 (from virtualenv)
  Downloading https://files.pythonhosted.org/packages/f5/0a/490fa011d699bb5a5f3a0cf57de82237f52a6db9d40f33c53b2736c9a1f9/distlib-0.3.1-py2.py3-none-any.whl (335kB)
Collecting typing-extensions>=3.6.4; python_version < "3.8" (from importlib-metadata>=0.12; python_version < "3.8"->virtualenv)
  Downloading https://files.pythonhosted.org/packages/60/7a/e881b5abb54db0e6e671ab088d079c57ce54e8a01a3ca443f561ccadb37e/typing_extensions-3.7.4.3-py3-none-any.whl
Collecting zipp>=0.5 (from importlib-metadata>=0.12; python_version < "3.8"->virtualenv)
  Downloading https://files.pythonhosted.org/packages/41/ad/6a4f1a124b325618a7fb758b885b68ff7b058eec47d9220a12ab38d90b1f/zipp-3.4.0-py3-none-any.whl
Installing collected packages: filelock, typing-extensions, zipp, importlib-metadata, appdirs, six, distlib, virtualenv
Successfully installed appdirs-1.4.4 distlib-0.3.1 filelock-3.0.12 importlib-metadata-3.4.0 six-1.15.0 typing-extensions-3.7.4.3 virtualenv-20.4.2 zipp-3.4.0
You are using pip version 18.1, however version 21.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

==============================================
II. Creating self-contained EBCLI installation
==============================================
created virtual environment CPython3.7.2.final.0-64 in 1841ms
  creator CPython3Posix(dest=/root/.ebcli-virtual-env, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/root/.local/share/virtualenv)
    added seed packages: pip==21.0.1, setuptools==52.0.0, wheel==0.36.2
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator
Collecting awsebcli
  Downloading awsebcli-3.19.3.tar.gz (249 kB)
Collecting botocore<1.20.0,>=1.19.0
  Downloading botocore-1.19.63-py2.py3-none-any.whl (7.2 MB)
Collecting cement==2.8.2
  Downloading cement-2.8.2.tar.gz (165 kB)
Collecting colorama<0.4.4,>=0.2.5
  Downloading colorama-0.4.3-py2.py3-none-any.whl (15 kB)
Collecting future<0.17.0,>=0.16.0
  Downloading future-0.16.0.tar.gz (824 kB)
Collecting pathspec==0.5.9
  Downloading pathspec-0.5.9.tar.gz (25 kB)
Collecting python-dateutil<3.0.0,>=2.1
  Downloading python_dateutil-2.8.1-py2.py3-none-any.whl (227 kB)
Collecting requests<=2.24,>=2.20.1
  Downloading requests-2.24.0-py2.py3-none-any.whl (61 kB)
Requirement already satisfied: setuptools>=20.0 in /root/.ebcli-virtual-env/lib/python3.7/site-packages (from awsebcli) (52.0.0)
Collecting setuptools>=20.0
  Using cached setuptools-53.0.0-py3-none-any.whl (784 kB)
Collecting semantic_version==2.5.0
  Downloading semantic_version-2.5.0-py3-none-any.whl (14 kB)
Collecting six<=1.15.0,>=1.12.0
  Using cached six-1.15.0-py2.py3-none-any.whl (10 kB)
Collecting termcolor==1.1.0
  Downloading termcolor-1.1.0.tar.gz (3.9 kB)
Collecting wcwidth<0.2.0,>=0.1.7
  Downloading wcwidth-0.1.9-py2.py3-none-any.whl (19 kB)
Collecting PyYAML<5.4,>=5.3.1
  Downloading PyYAML-5.3.1.tar.gz (269 kB)
Collecting urllib3<1.26,>=1.25.4
  Downloading urllib3-1.25.11-py2.py3-none-any.whl (127 kB)
Collecting docker-compose<1.26.0,>=1.25.2
  Downloading docker_compose-1.25.5-py2.py3-none-any.whl (139 kB)
Collecting blessed>=1.9.5
  Downloading blessed-1.17.12-py2.py3-none-any.whl (76 kB)
Collecting jmespath<1.0.0,>=0.7.1
  Downloading jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting cached-property<2,>=1.2.0
  Downloading cached_property-1.5.2-py2.py3-none-any.whl (7.6 kB)
Collecting texttable<2,>=0.9.0
  Downloading texttable-1.6.3-py2.py3-none-any.whl (10 kB)
Collecting dockerpty<1,>=0.4.1
  Downloading dockerpty-0.4.1.tar.gz (13 kB)
Collecting docker[ssh]<5,>=3.7.0
  Downloading docker-4.4.1-py2.py3-none-any.whl (146 kB)
Collecting jsonschema<4,>=2.5.1
  Downloading jsonschema-3.2.0-py2.py3-none-any.whl (56 kB)
Collecting docopt<1,>=0.6.1
  Downloading docopt-0.6.2.tar.gz (25 kB)
Collecting websocket-client<1,>=0.32.0
  Downloading websocket_client-0.57.0-py2.py3-none-any.whl (200 kB)
Collecting paramiko>=2.4.2
  Downloading paramiko-2.7.2-py2.py3-none-any.whl (206 kB)
Collecting attrs>=17.4.0
  Downloading attrs-20.3.0-py2.py3-none-any.whl (49 kB)
Collecting pyrsistent>=0.14.0
  Downloading pyrsistent-0.17.3.tar.gz (106 kB)
Collecting importlib-metadata
  Using cached importlib_metadata-3.4.0-py3-none-any.whl (10 kB)
Collecting cryptography>=2.5
  Downloading cryptography-3.4.1-cp36-abi3-manylinux2014_x86_64.whl (3.2 MB)
Collecting bcrypt>=3.1.3
  Downloading bcrypt-3.2.0-cp36-abi3-manylinux2010_x86_64.whl (63 kB)
Collecting pynacl>=1.0.1
  Downloading PyNaCl-1.4.0-cp35-abi3-manylinux1_x86_64.whl (961 kB)
Collecting cffi>=1.1
  Downloading cffi-1.14.4-cp37-cp37m-manylinux1_x86_64.whl (402 kB)
Collecting pycparser
  Downloading pycparser-2.20-py2.py3-none-any.whl (112 kB)
Collecting setuptools-rust>=0.11.4
  Downloading setuptools_rust-0.11.6-py3-none-any.whl (20 kB)
Collecting chardet<4,>=3.0.2
  Downloading chardet-3.0.4-py2.py3-none-any.whl (133 kB)
Collecting idna<3,>=2.5
  Downloading idna-2.10-py2.py3-none-any.whl (58 kB)
Collecting certifi>=2017.4.17
  Downloading certifi-2020.12.5-py2.py3-none-any.whl (147 kB)
Collecting toml>=0.9.0
  Downloading toml-0.10.2-py2.py3-none-any.whl (16 kB)
Collecting setuptools-rust>=0.11.4
  Downloading setuptools_rust-0.11.5-py3-none-any.whl (20 kB)
  Downloading setuptools_rust-0.11.4-py3-none-any.whl (20 kB)
INFO: pip is looking at multiple versions of idna to determine which version is compatible with other requirements. This could take a while.
Collecting idna<3,>=2.5
  Downloading idna-2.9-py2.py3-none-any.whl (58 kB)
  Downloading idna-2.8-py2.py3-none-any.whl (58 kB)
  Downloading idna-2.7-py2.py3-none-any.whl (58 kB)
  Downloading idna-2.6-py2.py3-none-any.whl (56 kB)
  Downloading idna-2.5-py2.py3-none-any.whl (55 kB)
INFO: pip is looking at multiple versions of chardet to determine which version is compatible with other requirements. This could take a while.
Collecting chardet<4,>=3.0.2
  Downloading chardet-3.0.3-py2.py3-none-any.whl (133 kB)
  Downloading chardet-3.0.2-py2.py3-none-any.whl (133 kB)
INFO: pip is looking at multiple versions of idna to determine which version is compatible with other requirements. This could take a while.
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. If you want to abort this run, you can press Ctrl + C to do so. To improve how pip performs, tell us what happened here: https://pip.pypa.io/surveys/backtracking
INFO: pip is looking at multiple versions of certifi to determine which version is compatible with other requirements. This could take a while.
Collecting certifi>=2017.4.17
  Downloading certifi-2020.11.8-py2.py3-none-any.whl (155 kB)
  Downloading certifi-2020.6.20-py2.py3-none-any.whl (156 kB)
  Downloading certifi-2020.4.5.2-py2.py3-none-any.whl (157 kB)
  Downloading certifi-2020.4.5.1-py2.py3-none-any.whl (157 kB)
  Downloading certifi-2020.4.5-py2.py3-none-any.whl (156 kB)
  Downloading certifi-2019.11.28-py2.py3-none-any.whl (156 kB)
  Downloading certifi-2019.9.11-py2.py3-none-any.whl (154 kB)
  Downloading certifi-2019.6.16-py2.py3-none-any.whl (157 kB)
  Downloading certifi-2019.3.9-py2.py3-none-any.whl (158 kB)
  Downloading certifi-2018.11.29-py2.py3-none-any.whl (154 kB)
  Downloading certifi-2018.10.15-py2.py3-none-any.whl (146 kB)
  Downloading certifi-2018.8.24-py2.py3-none-any.whl (147 kB)
  Downloading certifi-2018.8.13-py2.py3-none-any.whl (146 kB)
  Downloading certifi-2018.4.16-py2.py3-none-any.whl (150 kB)
  Downloading certifi-2018.1.18-py2.py3-none-any.whl (151 kB)
  Downloading certifi-2017.11.5-py2.py3-none-any.whl (330 kB)
  Downloading certifi-2017.7.27.1-py2.py3-none-any.whl (349 kB)
  Downloading certifi-2017.7.27-py2.py3-none-any.whl (349 kB)
  Downloading certifi-2017.4.17-py2.py3-none-any.whl (375 kB)
INFO: pip is looking at multiple versions of chardet to determine which version is compatible with other requirements. This could take a while.
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. If you want to abort this run, you can press Ctrl + C to do so. To improve how pip performs, tell us what happened here: https://pip.pypa.io/surveys/backtracking
INFO: pip is looking at multiple versions of certifi to determine which version is compatible with other requirements. This could take a while.
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. If you want to abort this run, you can press Ctrl + C to do so. To improve how pip performs, tell us what happened here: https://pip.pypa.io/surveys/backtracking
INFO: pip is looking at multiple versions of requests to determine which version is compatible with other requirements. This could take a while.
Collecting requests<=2.24,>=2.20.1
  Downloading requests-2.23.0-py2.py3-none-any.whl (58 kB)
  Downloading requests-2.22.0-py2.py3-none-any.whl (57 kB)
  Downloading requests-2.21.0-py2.py3-none-any.whl (57 kB)
  Downloading requests-2.20.1-py2.py3-none-any.whl (57 kB)
INFO: pip is looking at multiple versions of pyyaml to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of python-dateutil to determine which version is compatible with other requirements. This could take a while.
Collecting python-dateutil<3.0.0,>=2.1
  Downloading python_dateutil-2.8.0-py2.py3-none-any.whl (226 kB)
  Downloading python_dateutil-2.7.5-py2.py3-none-any.whl (225 kB)
  Downloading python_dateutil-2.7.4-py2.py3-none-any.whl (211 kB)
  Downloading python_dateutil-2.7.3-py2.py3-none-any.whl (211 kB)
  Downloading python_dateutil-2.7.2-py2.py3-none-any.whl (212 kB)
  Downloading python_dateutil-2.7.1-py2.py3-none-any.whl (212 kB)
  Downloading python_dateutil-2.7.0-py2.py3-none-any.whl (207 kB)
  Downloading python_dateutil-2.6.1-py2.py3-none-any.whl (194 kB)
  Downloading python_dateutil-2.6.0-py2.py3-none-any.whl (194 kB)
  Downloading python_dateutil-2.5.3-py2.py3-none-any.whl (201 kB)
  Downloading python_dateutil-2.5.2-py2.py3-none-any.whl (201 kB)
  Downloading python_dateutil-2.5.1-py2.py3-none-any.whl (200 kB)
  Downloading python_dateutil-2.5.0-py2.py3-none-any.whl (199 kB)
  Downloading python_dateutil-2.4.2-py2.py3-none-any.whl (188 kB)
  Downloading python-dateutil-2.4.1.post1.zip (212 kB)
WARNING: Discarding https://files.pythonhosted.org/packages/73/c4/d9e410b1641e210262123f49619070e46da2a7ede334cf6b6fb3db5ee985/python-dateutil-2.4.1.post1.zip#sha256=71357959bd98cbf425a6f5c5632826b548cb09d0ff3272a3565322f8aba58088 (from https://pypi.org/simple/python-dateutil/). Requested python-dateutil<3.0.0,>=2.1 from https://files.pythonhosted.org/packages/73/c4/d9e410b1641e210262123f49619070e46da2a7ede334cf6b6fb3db5ee985/python-dateutil-2.4.1.post1.zip#sha256=71357959bd98cbf425a6f5c5632826b548cb09d0ff3272a3565322f8aba58088 (from awsebcli) has inconsistent version: filename has '2.4.1.post1', but metadata has '2.4.1'
  Downloading python-dateutil-2.4.1.post1.tar.gz (207 kB)
WARNING: Discarding https://files.pythonhosted.org/packages/9c/b0/5948496efa852dfa78751c3f494f57fa01bfc453b4a7b7b47b0c2e0b6a80/python-dateutil-2.4.1.post1.tar.gz#sha256=aa9bdbd60c395db90204609f1fb5aeb3797870f65c09f04f243476d22f8f4615 (from https://pypi.org/simple/python-dateutil/). Requested python-dateutil<3.0.0,>=2.1 from https://files.pythonhosted.org/packages/9c/b0/5948496efa852dfa78751c3f494f57fa01bfc453b4a7b7b47b0c2e0b6a80/python-dateutil-2.4.1.post1.tar.gz#sha256=aa9bdbd60c395db90204609f1fb5aeb3797870f65c09f04f243476d22f8f4615 (from awsebcli) has inconsistent version: filename has '2.4.1.post1', but metadata has '2.4.1'
  Downloading python_dateutil-2.4.1-py2.py3-none-any.whl (188 kB)
  Downloading python_dateutil-2.4.0-py2.py3-none-any.whl (175 kB)
  Downloading python_dateutil-2.3-py2.py3-none-any.whl (173 kB)
  Downloading python-dateutil-2.2.tar.gz (259 kB)
  Downloading python-dateutil-2.1.tar.gz (152 kB)
INFO: pip is looking at multiple versions of requests to determine which version is compatible with other requirements. This could take a while.
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. If you want to abort this run, you can press Ctrl + C to do so. To improve how pip performs, tell us what happened here: https://pip.pypa.io/surveys/backtracking
INFO: pip is looking at multiple versions of pyyaml to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of python-dateutil to determine which version is compatible with other requirements. This could take a while.
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. If you want to abort this run, you can press Ctrl + C to do so. To improve how pip performs, tell us what happened here: https://pip.pypa.io/surveys/backtracking
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. If you want to abort this run, you can press Ctrl + C to do so. To improve how pip performs, tell us what happened here: https://pip.pypa.io/surveys/backtracking
INFO: pip is looking at multiple versions of pyrsistent to determine which version is compatible with other requirements. This could take a while.
Collecting pyrsistent>=0.14.0
  Downloading pyrsistent-0.16.1.tar.gz (108 kB)
  Downloading pyrsistent-0.16.0.tar.gz (108 kB)
  Downloading pyrsistent-0.15.7.tar.gz (107 kB)
  Downloading pyrsistent-0.15.6.tar.gz (107 kB)
  Downloading pyrsistent-0.15.5.tar.gz (107 kB)
  Downloading pyrsistent-0.15.4.tar.gz (107 kB)
  Downloading pyrsistent-0.15.3.tar.gz (106 kB)
  Downloading pyrsistent-0.15.2.tar.gz (106 kB)
  Downloading pyrsistent-0.15.1.tar.gz (106 kB)
  Downloading pyrsistent-0.15.0.tar.gz (105 kB)
  Downloading pyrsistent-0.14.11.tar.gz (104 kB)
  Downloading pyrsistent-0.14.10.tar.gz (104 kB)
  Downloading pyrsistent-0.14.9.tar.gz (104 kB)
  Downloading pyrsistent-0.14.8.tar.gz (104 kB)
  Downloading pyrsistent-0.14.7.tar.gz (103 kB)
  Downloading pyrsistent-0.14.6.tar.gz (100 kB)
  Downloading pyrsistent-0.14.5.tar.gz (99 kB)
  Downloading pyrsistent-0.14.4.tar.gz (98 kB)
  Downloading pyrsistent-0.14.3.tar.gz (98 kB)
  Downloading pyrsistent-0.14.2.tar.gz (97 kB)
  Downloading pyrsistent-0.14.1.tar.gz (97 kB)
  Downloading pyrsistent-0.14.0.tar.gz (96 kB)
Pulling docker image gitlab/gitlab-runner-helper:x86_64-05161b14 ...
ERROR: Job failed: execution took longer than 1h0m0s seconds
gangulymanish commented 3 years ago

Facing same issue, any idea how to fix this ?

javiercanillas commented 3 years ago

Another github thread with useful information: https://github.com/aws/aws-elastic-beanstalk-cli/issues/61

tonanuvem commented 3 years ago

Same problem for me.

Tip from the other thread, although this was enough of a temporary fix:

pip3 install awsebcli cryptography==3.3.1

avoup commented 3 years ago

installing an older version fixed the problem:

python scripts/ebcli_installer.py --version 3.14.13

JeroenVdb commented 3 years ago

I have the same problem. My docker build is now running for 3 hours.

Using an old version like @avoup suggested works but this doesn't seem like a great fix. Is this something with awebcli itself?

javiercanillas commented 3 years ago

@JeroenVdb I think this was already fixed some time ago. I'm using: 

    # Setting python to version 3.7.1
    pyenv global 3.7.1
    # move pip from 10.X to latest
    pip install --upgrade pip
    # install awsebcli
    pip install --upgrade awsebcli==3.19.4

I hope this helps.

JeroenVdb commented 3 years ago

I'm a bit "stuck" with a newer Python 3.9 (latest Alpine release). @javiercanillas is using Python 3.7.1 the krux in solving this issue?