Connection issues when downloading MMseqs2 via wget

piehld commented 1 year ago

Expected Behavior

Downloading the latest MMseqs2 tarball with wget should work consistently: wget https://mmseqs.com/latest/mmseqs-linux-avx2.tar.gz

Current Behavior

Downloading the latest MMseqs2 tarball with wget is not working consistently, nor for curl (see below for examples and output).

Steps to Reproduce (for bugs)

First test case—running wget doesn't work the first time, but works the second time:


root@linux-server:/opt/test# wget https://mmseqs.com/latest/mmseqs-linux-avx2.tar.gz
--2023-05-22 08:17:43--  https://mmseqs.com/latest/mmseqs-linux-avx2.tar.gz
Resolving mmseqs.com (mmseqs.com)... 147.46.145.74
Connecting to mmseqs.com (mmseqs.com)|147.46.145.74|:443... connected.
Unable to establish SSL connection.

root@linux-server:/opt/test# wget https://mmseqs.com/latest/mmseqs-linux-avx2.tar.gz --2023-05-22 08:18:04-- https://mmseqs.com/latest/mmseqs-linux-avx2.tar.gz Resolving mmseqs.com (mmseqs.com)... 147.46.145.74 Connecting to mmseqs.com (mmseqs.com)|147.46.145.74|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 44786049 (43M) [application/octet-stream] Saving to: 'mmseqs-linux-avx2.tar.gz'

mmseqs-linux-avx2.tar.gz 100%[====================================================================================================================================================>] 42.71M 10.3MB/s in 4.5s

2023-05-22 08:18:10 (9.44 MB/s) - 'mmseqs-linux-avx2.tar.gz' saved [44786049/44786049]


2. Second test case—using different `--secure-protocol` settings also provides inconsistent success (in trying to follow advice from [here](https://serverfault.com/a/622127)):

ubuntu@linux-server:~$ wget --secure-protocol=TLSv1_3 https://mmseqs.com/latest/mmseqs-linux-avx2.tar.gz --2023-05-22 07:53:35-- https://mmseqs.com/latest/mmseqs-linux-avx2.tar.gz Resolving mmseqs.com (mmseqs.com)... 147.46.145.74 Connecting to mmseqs.com (mmseqs.com)|147.46.145.74|:443... connected. Unable to establish SSL connection.

ubuntu@linux-server:~$ wget --secure-protocol=TLSv1_2 https://mmseqs.com/latest/mmseqs-linux-avx2.tar.gz --2023-05-22 07:54:12-- https://mmseqs.com/latest/mmseqs-linux-avx2.tar.gz Resolving mmseqs.com (mmseqs.com)... 147.46.145.74 Connecting to mmseqs.com (mmseqs.com)|147.46.145.74|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 44786049 (43M) [application/octet-stream] Saving to: 'mmseqs-linux-avx2.tar.gz.3'

mmseqs-linux-avx2.tar.gz.3 100%[=======================================================================================>] 42.71M 11.0MB/s in 4.1s

2023-05-22 07:54:17 (10.5 MB/s) - 'mmseqs-linux-avx2.tar.gz.3' saved [44786049/44786049]

ubuntu@linux-server:~$ wget --secure-protocol=TLSv1_3 https://mmseqs.com/latest/mmseqs-linux-avx2.tar.gz --2023-05-22 07:55:19-- https://mmseqs.com/latest/mmseqs-linux-avx2.tar.gz Resolving mmseqs.com (mmseqs.com)... 147.46.145.74 Connecting to mmseqs.com (mmseqs.com)|147.46.145.74|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 44786049 (43M) [application/octet-stream] Saving to: 'mmseqs-linux-avx2.tar.gz.4'

mmseqs-linux-avx2.tar.gz.4 100%[=======================================================================================>] 42.71M 11.2MB/s in 4.0s

2023-05-22 07:55:24 (10.7 MB/s) - 'mmseqs-linux-avx2.tar.gz.4' saved [44786049/44786049]


3. Third test case—using `curl` also results in inconsistent success:

root@linux-server:/opt/test# curl -LO https://mmseqs.com/latest/mmseqs-linux-avx2.tar.gz % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 curl: (35) OpenSSL SSL_connect: Connection reset by peer in connection to mmseqs.com:443 root@linux-server:/opt/test# curl -LO https://mmseqs.com/latest/mmseqs-linux-avx2.tar.gz % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 42.7M 100 42.7M 0 0 9058k 0 0:00:04 0:00:04 --:--:-- 9904k


## Context
We have been observing this issue for the past two weeks, which has been arising in one of our package's Azure Pipelines tests. The examples above were reproduced using an Ubuntu server.

## Your Environment
* Which MMseqs version was used (Statically-compiled, self-compiled, Homebrew, etc.): `latest` static version
* Operating system and version:
  * For example cases above:

Linux 5.4.0-139-generic #156-Ubuntu SMP x86_64 x86_64 x86_64 GNU/Linux

  * Azure image:

Operating System Ubuntu 20.04.6 LTS Runner Image Image: ubuntu-20.04 Version: 20230507.1 Included Software: https://github.com/actions/runner-images/blob/ubuntu20/20230507.1/images/linux/Ubuntu2004-Readme.md Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu20%2F20230507.1

milot-mirdita commented 1 year ago

I have recently migrated the binary server to a different machine since the old one was full. The new server seems to have the issue that the first requests fails and then subsequent requests don't seem to have any issues.

I think I have to coordinate with our University IT, I don't see any reason on the server why this should be happening.

To fix your CI for now I would recommend to add curl retry flags curl --retry 5 --retry-all-errors (or the alternative wget flags).

Sorry about this :/

piehld commented 1 year ago

Thank you for the fast response and suggestion for a temporary fix. Although, the --retry* flags for both curl and wget don't appear to apply for SSL-related connection errors, so neither will retry in this situation. I think it's necessary to perform the retries at the outer-scope—i.e., try the download and check for the file locally.

milot-mirdita commented 1 year ago

From what I can tell this is not actually an SSL issue. Something in our network completely eats the first packet, and nothing appears on the server. Subsequent requests appear correctly in packet dumps.

soedinglab / MMseqs2