search

plasmabio / plasma

Plasma is an e-learning Jupyter-based platform for data analysis
https://docs.plasmabio.org
BSD 3-Clause "New" or "Revised" License
42 stars 12 forks source link

ansible task without https stuck #85

Closed pierrepo closed 4 years ago

pierrepo commented 4 years ago

While deploying without https, ansible seems stuck (at least for more than half an hour):

$ ansible-playbook all.yml -i hosts -u ubuntu --tags "all,no-https"

PLAY [all] *************************************************************************************************************************************************

TASK [Gathering Facts] *************************************************************************************************************************************
ok: [51.68.113.239]

TASK [Install aptitude using apt] **************************************************************************************************************************
changed: [51.68.113.239]

TASK [Install required system packages] ********************************************************************************************************************
changed: [51.68.113.239] => (item=apt-transport-https)
ok: [51.68.113.239] => (item=ca-certificates)
ok: [51.68.113.239] => (item=curl)
changed: [51.68.113.239] => (item=software-properties-common)
changed: [51.68.113.239] => (item=python3-pip)
changed: [51.68.113.239] => (item=virtualenv)
ok: [51.68.113.239] => (item=python3-setuptools)

TASK [Add Docker GPG apt Key] ******************************************************************************************************************************
changed: [51.68.113.239]

TASK [Add Docker Repository] *******************************************************************************************************************************
changed: [51.68.113.239]

TASK [Update apt and install docker-ce] ********************************************************************************************************************
changed: [51.68.113.239]

TASK [Install Docker Module for Python] ********************************************************************************************************************
changed: [51.68.113.239]

PLAY [all] *************************************************************************************************************************************************

TASK [Gathering Facts] *************************************************************************************************************************************
pierrepo commented 4 years ago

At some point it stopped. Re-running the previous command led to completion. Weird.

$ ansible-playbook all.yml -i hosts -u ubuntu --tags "all,no-https"

PLAY [all] *************************************************************************************************************************************************

TASK [Gathering Facts] *************************************************************************************************************************************
ok: [51.68.113.239]

TASK [Install aptitude using apt] **************************************************************************************************************************
ok: [51.68.113.239]

TASK [Install required system packages] ********************************************************************************************************************
ok: [51.68.113.239] => (item=apt-transport-https)
ok: [51.68.113.239] => (item=ca-certificates)
ok: [51.68.113.239] => (item=curl)
ok: [51.68.113.239] => (item=software-properties-common)
ok: [51.68.113.239] => (item=python3-pip)
ok: [51.68.113.239] => (item=virtualenv)
ok: [51.68.113.239] => (item=python3-setuptools)

TASK [Add Docker GPG apt Key] ******************************************************************************************************************************
ok: [51.68.113.239]

TASK [Add Docker Repository] *******************************************************************************************************************************
ok: [51.68.113.239]

TASK [Update apt and install docker-ce] ********************************************************************************************************************
ok: [51.68.113.239]

TASK [Install Docker Module for Python] ********************************************************************************************************************
ok: [51.68.113.239]

PLAY [all] *************************************************************************************************************************************************

TASK [Gathering Facts] *************************************************************************************************************************************
ok: [51.68.113.239]

TASK [Install extra system packages] ***********************************************************************************************************************
changed: [51.68.113.239] => (item=jq)

TASK [Install ctop] ****************************************************************************************************************************************
changed: [51.68.113.239]

PLAY [all] *************************************************************************************************************************************************

TASK [Gathering Facts] *************************************************************************************************************************************
ok: [51.68.113.239]

TASK [Add the jovyan user] *********************************************************************************************************************************
changed: [51.68.113.239]

TASK [Add foo Test User] ***********************************************************************************************************************************
changed: [51.68.113.239]

TASK [Add bar Test User] ***********************************************************************************************************************************
changed: [51.68.113.239]

PLAY [all] *************************************************************************************************************************************************

TASK [Gathering Facts] *************************************************************************************************************************************
ok: [51.68.113.239]

TASK [Install aptitude using apt] **************************************************************************************************************************
ok: [51.68.113.239]

TASK [Install required system packages] ********************************************************************************************************************
ok: [51.68.113.239] => (item=curl)
ok: [51.68.113.239] => (item=python3)
ok: [51.68.113.239] => (item=python3-dev)
ok: [51.68.113.239] => (item=python3-pip)

TASK [Download the TLJH installer] *************************************************************************************************************************
changed: [51.68.113.239]

TASK [Upgrade the tljh-plasmabio plugin] *******************************************************************************************************************
fatal: [51.68.113.239]: FAILED! => {"changed": true, "cmd": "/opt/tljh/hub/bin/pip3 install --upgrade git+https://github.com/plasmabio/plasmabio@master#\"egg=tljh-plasmabio&subdirectory=tljh-plasmabio\"", "delta": "0:00:00.006342", "end": "2020-04-08 16:30:49.515537", "msg": "non-zero return code", "rc": 127, "start": "2020-04-08 16:30:49.509195", "stderr": "/bin/sh: 1: /opt/tljh/hub/bin/pip3: not found", "stderr_lines": ["/bin/sh: 1: /opt/tljh/hub/bin/pip3: not found"], "stdout": "", "stdout_lines": []}
...ignoring

TASK [Run the TLJH installer] ******************************************************************************************************************************
changed: [51.68.113.239]

TASK [Copy the config] *************************************************************************************************************************************
changed: [51.68.113.239]

TASK [Reload the hub] **************************************************************************************************************************************
changed: [51.68.113.239]

TASK [Disable HTTPS] ***************************************************************************************************************************************
changed: [51.68.113.239]

TASK [Reload the proxy] ************************************************************************************************************************************
changed: [51.68.113.239]

TASK [Pull jupyter/repo2docker] ****************************************************************************************************************************
changed: [51.68.113.239]

PLAY RECAP *************************************************************************************************************************************************
51.68.113.239              : ok=25   changed=13   unreachable=0    failed=0    skipped=0    rescued=0    ignored=1
jtpio commented 4 years ago

Looking at the first log, it seems that it had just finished with the docker playbook and was about to start with the TLJH playbook.

This might just have been the result of a connection loss or a hiccup on the VM.

jtpio commented 4 years ago

Let me know if that's still the case after moving the HTTPS setup to its own playbook.

pierrepo commented 4 years ago

The issue is still here. This command hanged

$ ansible-playbook site.yml -i hosts -u ubuntu

at step

TASK [Install Docker Module for Python] *****************************************************************************************************
changed: [51.68.113.239]

PLAY [all] **********************************************************************************************************************************

TASK [Gathering Facts] **********************************************************************************************************************

I had to stop (Ctrl + C) and rerun it (with success).

It seems the deployment hanged between docker.yml and utils.yml playbooks. It's weird because utils.yml is not doing much.

jtpio commented 4 years ago

@pierrepo it might be worth trying with the following config in ~/.ssh/config:

Host *
   ServerAliveInterval 60
   ServerAliveCountMax 10

To check whether it's related to a connection loss during the execution of the playbook.

jtpio commented 4 years ago

Just tried with the above settings on a fresh install.

It took ~12 minutes to run site.yml, but completed successfully without connection drops.

pierrepo commented 4 years ago

The test worked like a charm! About ~ 12 minutes this time. The SSH parameters probably helped! Thanks. I'm closing this issue.