Installation via apt on Ubuntu 20.04 extremely slow

elmonitis commented 3 years ago

Trying to install Zotero on my Ubuntu 20.04 laptop via apt on a hi-speed connection. The progress indicator says the download will complete in about 2 hours, sometimes it would change to about 1 hour or so, at other times it switches to 5 hours or so. I'm installing Zotero alone and nothing else.

Occasionally, and after some time, I'd get the following:

Get:1 https://github.com/retorquere/zotero-deb/releases/download/apt-get ./ zotero 5.0.93 [63.4 MB]
Err:1 https://github.com/retorquere/zotero-deb/releases/download/apt-get ./ zotero 5.0.93                                                                            
  403  Forbidden [IP: 52.217.81.252 443]
E: Failed to fetch https://github-production-release-asset-2e65be.s3.amazonaws.com/153281752/c5af7380-25cf-11eb-81ef-6a4dbbc98d95?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A/20201125/us-east-1/s3/aws4_request&X-Amz-Date=20201125T012350Z&X-Amz-Expires=300&X-Amz-Signature=de9846bd6e046cd81b3fa1ea23f6fbb251a9935cb62eee2471625d3fdb5893ed&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=153281752&response-content-disposition=attachment; filename=zotero_5.0.93_amd64.deb&response-content-type=application/octet-stream  403  Forbidden [IP: 52.217.81.252 443]
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?

In that case, I just restart the download with sudo apt install zotero.

Right now, after about 30 minutes of downloading, total download is still at 33%, and the progress indicator is saying 1 h 58 m.

Any help much appreciated.

retorquere commented 3 years ago

What does

time curl -vLO https://github.com/retorquere/zotero-deb/releases/download/apt-get/jurism_5.0.93m3_amd64.deb

show for you?

elmonitis commented 3 years ago

Update: The download and installation of Zotero finished finally after about 4 hours.

elmonitis commented 3 years ago

What does

time curl -vLO https://github.com/retorquere/zotero-deb/releases/download/apt-get/jurism_5.0.93m3_amd64.deb

show for you?

This is what I see:

real    0m21.433s
user    0m0.640s
sys 0m0.904s

retorquere commented 3 years ago

That can't be all the output, but that time doesn't seem out of whack. This is on the system where the install doesn't work? Because that's the request that seems to be stalling for you.

elmonitis commented 3 years ago

I ran the timed curl command on my desktop. There were several lines above those three that I pasted. I'll re-run the command on my laptop (which was where the installation stalled in the first place). For what it's worth, after Zotero finished installing on my laptop earlier, I ran sudo apt update and then sudo apt upgrade and the whole thing finished in less than 2 minutes, with 5 packages upgraded. Will post the results from my laptop in a bit.

elmonitis commented 3 years ago

That can't be all the output, but that time doesn't seem out of whack. This is on the system where the install doesn't work? Because that's the request that seems to be stalling for you.

Here's the output for the command executed on my laptop (connected via wired connection):

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 192.30.255.112:443...
* TCP_NODELAY set
* Connected to github.com (192.30.255.112) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
} [5 bytes data]
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
* TLSv1.3 (IN), TLS handshake, Server hello (2):
{ [122 bytes data]
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
{ [25 bytes data]
* TLSv1.3 (IN), TLS handshake, Certificate (11):
{ [2976 bytes data]
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
{ [264 bytes data]
* TLSv1.3 (IN), TLS handshake, Finished (20):
{ [36 bytes data]
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
} [1 bytes data]
* TLSv1.3 (OUT), TLS handshake, Finished (20):
} [36 bytes data]
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: C=US; ST=California; L=San Francisco; O=GitHub, Inc.; CN=github.com
*  start date: May  5 00:00:00 2020 GMT
*  expire date: May 10 12:00:00 2022 GMT
*  subjectAltName: host "github.com" matched cert's "github.com"
*  issuer: C=US; O=DigiCert Inc; OU=www.digicert.com; CN=DigiCert SHA2 High Assurance Server CA
*  SSL certificate verify ok.
} [5 bytes data]
> GET /retorquere/zotero-deb/releases/download/apt-get/jurism_5.0.93m3_amd64.deb HTTP/1.1
> Host: github.com
> User-Agent: curl/7.68.0
> Accept: */*
> 
{ [5 bytes data]
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
{ [57 bytes data]
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
{ [57 bytes data]
* old SSL session ID is stale, removing
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0{ [5 bytes data]
* Mark bundle as not supporting multiuse
< HTTP/1.1 302 Found
< date: Wed, 25 Nov 2020 11:56:29 GMT
< content-type: text/html; charset=utf-8
< server: GitHub.com
< status: 302 Found
< vary: X-PJAX, Accept-Encoding, Accept, X-Requested-With, Accept-Encoding
< location: https://github-production-release-asset-2e65be.s3.amazonaws.com/153281752/c34d1980-25cf-11eb-8588-afe2e7b13dcd?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20201125%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20201125T115629Z&X-Amz-Expires=300&X-Amz-Signature=6463604f8e65997d1ba74efa66225b4e3d015e93ec765f9588764690e53e92c3&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=153281752&response-content-disposition=attachment%3B%20filename%3Djurism_5.0.93m3_amd64.deb&response-content-type=application%2Foctet-stream
< cache-control: no-cache
< strict-transport-security: max-age=31536000; includeSubdomains; preload
< x-frame-options: deny
< x-content-type-options: nosniff
< x-xss-protection: 1; mode=block
< referrer-policy: no-referrer-when-downgrade
< expect-ct: max-age=2592000, report-uri="https://api.github.com/_private/browser/errors"
{ [5 bytes data]
< content-security-policy: default-src 'none'; base-uri 'self'; block-all-mixed-content; connect-src 'self' uploads.github.com www.githubstatus.com collector.githubapp.com api.github.com github-cloud.s3.amazonaws.com github-production-repository-file-5c1aeb.s3.amazonaws.com github-production-upload-manifest-file-7fdce7.s3.amazonaws.com github-production-user-asset-6210df.s3.amazonaws.com cdn.optimizely.com logx.optimizely.com/v1/events wss://alive.github.com; font-src github.githubassets.com; form-action 'self' github.com gist.github.com; frame-ancestors 'none'; frame-src render.githubusercontent.com; img-src 'self' data: github.githubassets.com identicons.github.com collector.githubapp.com github-cloud.s3.amazonaws.com *.githubusercontent.com; manifest-src 'self'; media-src 'none'; script-src github.githubassets.com; style-src 'unsafe-inline' github.githubassets.com; worker-src github.com/socket-worker.js gist.github.com/socket-worker.js
< Set-Cookie: _gh_sess=65yssR1s6pkPAkF1tSFNXgAeJ13ty%2BoywMbvn1ft3SAWWya%2BlXSca2CzKM2QFs0LjrVa4ZtZ8LQ01ds6c0Fgsg%2BJfTBE3Gzymmy5M66koo5xbLX0tmouvHosGZHRODM09q3EAx0DdbAZoCFQiSgXYAL5UCOfOFsY41hU8mX1KmR72iG8sis%2BFSrfA49zH153O7FXL6xtzvnEzs5LNeRvyqjtGPFWDGSZSwkG94DvV7qwten8NOfQM5ln3xUPykCD8TNgQgjkr5MrT8Lr9Xeprg%3D%3D--mWVIO92LRcNk53ZY--i9j4Ra05BmDCm5Tzm%2BlVeA%3D%3D; Path=/; HttpOnly; Secure; SameSite=Lax
< Set-Cookie: _octo=GH1.1.1123279603.1606305389; Path=/; Domain=github.com; Expires=Thu, 25 Nov 2021 11:56:29 GMT; Secure; SameSite=Lax
< Set-Cookie: logged_in=no; Path=/; Domain=github.com; Expires=Thu, 25 Nov 2021 11:56:29 GMT; HttpOnly; Secure; SameSite=Lax
< Content-Length: 651
{ [5 bytes data]
< X-GitHub-Request-Id: 3C18:80E4:81C92F:A88950:5FBE466C
< 
* Ignoring the response-body
{ [651 bytes data]
100   651  100   651    0     0    721      0 --:--:-- --:--:-- --:--:--   721
* Connection #0 to host github.com left intact
* Issue another request to this URL: 'https://github-production-release-asset-2e65be.s3.amazonaws.com/153281752/c34d1980-25cf-11eb-8588-afe2e7b13dcd?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20201125%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20201125T115629Z&X-Amz-Expires=300&X-Amz-Signature=6463604f8e65997d1ba74efa66225b4e3d015e93ec765f9588764690e53e92c3&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=153281752&response-content-disposition=attachment%3B%20filename%3Djurism_5.0.93m3_amd64.deb&response-content-type=application%2Foctet-stream'
*   Trying 52.217.77.148:443...
* TCP_NODELAY set
* Connected to github-production-release-asset-2e65be.s3.amazonaws.com (52.217.77.148) port 443 (#1)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
} [5 bytes data]
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
* TLSv1.3 (IN), TLS handshake, Server hello (2):
{ [91 bytes data]
* TLSv1.2 (IN), TLS handshake, Certificate (11):
{ [2900 bytes data]
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
{ [333 bytes data]
* TLSv1.2 (IN), TLS handshake, Server finished (14):
{ [4 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
} [70 bytes data]
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
} [1 bytes data]
* TLSv1.2 (OUT), TLS handshake, Finished (20):
} [16 bytes data]
* TLSv1.2 (IN), TLS handshake, Finished (20):
{ [16 bytes data]
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: C=US; ST=Washington; L=Seattle; O=Amazon.com, Inc.; CN=*.s3.amazonaws.com
*  start date: Nov  9 00:00:00 2019 GMT
*  expire date: Mar 12 12:00:00 2021 GMT
*  subjectAltName: host "github-production-release-asset-2e65be.s3.amazonaws.com" matched cert's "*.s3.amazonaws.com"
*  issuer: C=US; O=DigiCert Inc; OU=www.digicert.com; CN=DigiCert Baltimore CA-2 G2
*  SSL certificate verify ok.
  0     0    0     0    0     0      0      0 --:--:--  0:00:01 --:--:--     0} [5 bytes data]
> GET /153281752/c34d1980-25cf-11eb-8588-afe2e7b13dcd?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20201125%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20201125T115629Z&X-Amz-Expires=300&X-Amz-Signature=6463604f8e65997d1ba74efa66225b4e3d015e93ec765f9588764690e53e92c3&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=153281752&response-content-disposition=attachment%3B%20filename%3Djurism_5.0.93m3_amd64.deb&response-content-type=application%2Foctet-stream HTTP/1.1
> Host: github-production-release-asset-2e65be.s3.amazonaws.com
> User-Agent: curl/7.68.0
> Accept: */*
> 
{ [5 bytes data]
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< x-amz-id-2: vL+lVb3wqjMnuVVka/y0CeWbQjH2AeR+sbYS6aXXSnfhUQYAiZQacbDkTABjbNQNDdt3uQn3lkQ=
< x-amz-request-id: 9J7NBQEH9N5QCPEJ
< Date: Wed, 25 Nov 2020 11:56:31 GMT
< Last-Modified: Fri, 13 Nov 2020 15:46:25 GMT
< ETag: "c04bf6342ccec5918c96efa870173bcc"
< Content-Disposition: attachment; filename=jurism_5.0.93m3_amd64.deb
< Accept-Ranges: bytes
< Content-Type: application/octet-stream
< Content-Length: 65030118
< Server: AmazonS3
< 
{ [5 bytes data]
100 62.0M  100 62.0M    0     0  3100k      0  0:00:20  0:00:20 --:--:-- 3352k
* Connection #1 to host github-production-release-asset-2e65be.s3.amazonaws.com left intact

real    0m20.524s
user    0m0.751s
sys 0m1.083s

retorquere commented 3 years ago

Nothing really weird in that, but that's the request apt puts out. I'm not sure why apt is so slow when curl fetches it much faster, on the same system. It's not the connection itself, obviously.

Other packages can indeed update faster, they're coming from different servers. I'm at a loss, I don't know how to debug apt.

elmonitis commented 3 years ago

Nothing really weird in that, but that's the request apt puts out. I'm not sure why apt is so slow when curl fetches it much faster, on the same system. It's not the connection itself, obviously.

Other packages can indeed update faster, they're coming from different servers. I'm at a loss, I don't know how to debug apt.

I also tried downloading the Zotero DEB file yesterday via Chrome (still on the laptop). Same thing. Really slow download speed. Estimated time was around 2 hours or so. I just canceled the download.

retorquere commented 3 years ago

I just tried that, and downloading the deb with Chrome takes the approx 25 secs I expect.

elmonitis commented 3 years ago

I also did the same right now using my desktop, but the same Internet connection. Same speed as yours — about 25 seconds to download 60.4 MB of the deb file.

Maybe it was just a temporary network lag from my end yesterday.

retorquere commented 3 years ago

And apt?

elmonitis commented 3 years ago

I didn't try apt on my desktop because I already have Zotero installed on it. I tried reinstalling on my laptop using apt. It took less than a minute this time.

retorquere commented 3 years ago

I have absolutely no idea what the cause could be 🤷 but it must be something between the GH servers and your systems. I promise you I am not vetting and stalling individual downloads 😀 .

elmonitis commented 3 years ago

Neither do I, but, as you said, it must have been a connection problem between GH and my end because it also affected downloading the DEB file through Chrome. Or maybe some issue with my ISP.

Since this mystery has resolved on its own, I lay this issue to rest. Thank you for the help @retorquere.

retorquere / zotero-deb

Installation via apt on Ubuntu 20.04 extremely slow #34