astrojuanlu
commented
5 years ago Interestingly, when I switched from the router to a WiFi hotspot with my smartphone the problem disappeared. So now I suspect there is something wrong with my ISP...
Closed astrojuanlu closed 5 years ago
astrojuanlu
commented
5 years ago Interestingly, when I switched from the router to a WiFi hotspot with my smartphone the problem disappeared. So now I suspect there is something wrong with my ISP...
di
commented
5 years ago @Juanlu001 Who is your ISP?
astrojuanlu
commented
5 years ago Sorry, somehow my fastly-debug output didn't make to the report. It's CABLEWORLD-AS, ES
ewogICJnZW9pcCI6IHsKICAgICJjaSI6ICJlbHgiLAogICAgInN0IjogIkEiLAogICAgImN0IjogInNwYWluIiwKICAgICJjbyI6ICJFVSIsCiAgICAiY19hc24iOiAiMzUzOTQiLAogICAgImNfYXNuX25hbWUiOiAidGVsZXZpZGVvIG5vdmVsZGEgcy5hLiB1bmlwZXJzb25hbCIsCiAgICAicl9pcCI6ICI3NC4xMjUuNDcuMTM2IiwKICAgICJyX2FzbiI6ICIxNTE2OSIsCiAgICAicl9hc25fbmFtZSI6ICJnb29nbGUgbGxjIiwKICAgICJyX2NpIjogImJydXNzZWxzIiwKICAgICJyX3N0IjogIkJSVSIsCiAgICAicl9jdCI6ICJiZWxnaXVtIiwKICAgICJyX2NvIjogIkVVIgogIH0sCiAgInBvcExhdGVuY3kiOiB7CiAgICAibGhyIjogNDEsCiAgICAibGN5IjogNDEsCiAgICAiY2RnIjogMzMsCiAgICAibWFkIjogMTUsCiAgICAiZnJhIjogNDUsCiAgICAiaGhuIjogNDYsCiAgICAiYW1zIjogNDYsCiAgICAiYm1hIjogNzAsCiAgICAiaGVsIjogNzQsCiAgICAiamZrIjogMTE4LAogICAgImhrZyI6IDE4OCwKICAgICJhbnkiOiAxNQogIH0sCiAgInBvcEFzc2lnbm1lbnRzIjogewogICAgImFjIjogIm1hZCIsCiAgICAiYXMiOiAibWFkIgogIH0sCiAgInJlcXVlc3QiOiB7CiAgICAicmVzb2x2ZXJfaXAiOiAiNzQuMTI1LjczLjY3IiwKICAgICJyZXNvbHZlcl9hc19uYW1lIjogIkdPT0dMRSAtIEdvb2dsZSBMTEMsIFVTIiwKICAgICJyZXNvbHZlcl9hc19udW1iZXIiOiAiMTUxNjkiLAogICAgInJlc29sdmVyX2NvdW50cnlfY29kZSI6ICJVUyIsCiAgICAiY2xpZW50X2lwIjogIjE4NS4xNzIuMTcwLjE2MCIsCiAgICAiY2xpZW50X2FzX25hbWUiOiAiQ0FCTEVXT1JMRC1BUywgRVMiLAogICAgImNsaWVudF9hc19udW1iZXIiOiAiMzUzOTQiLAogICAgInRpbWUiOiAiMjAxOS0wMS0xNlQxODo0MToxNS4wMDBaIiwKICAgICJob3N0IjogInd3dy5mYXN0bHktZGVidWcuY29tIiwKICAgICJhY2NlcHQiOiAidGV4dC9odG1sLGFwcGxpY2F0aW9uL3hodG1sK3htbCxhcHBsaWNhdGlvbi94bWw7cT0wLjksKi8qO3E9MC44IiwKICAgICJ1c2VyYWdlbnQiOiAiTW96aWxsYS81LjAgKFgxMTsgVWJ1bnR1OyBMaW51eCB4ODZfNjQ7IHJ2OjY0LjApIEdlY2tvLzIwMTAwMTAxIEZpcmVmb3gvNjQuMCIsCiAgICAiYWNjZXB0bGFuZ3VhZ2UiOiAiZW4tVVMsZW47cT0wLjUiLAogICAgImFjY2VwdGVuY29kaW5nIjogImd6aXAiLAogICAgImZhc3RseXNlcnZlcmlwIjogIjE1MS4xMDEuMTkyLjY0IiwKICAgICJ4ZmYiOiAiMTg1LjE3Mi4xNzAuMTYwIiwKICAgICJkYXRhY2VudGVyIjogIk1BRCIsCiAgICAiYmFuZHdpZHRoX21icHMiOiAiMjAuNTYiLAogICAgImN3bmQiOiAyNiwKICAgICJuZXh0aG9wIjogIjE3Mi4xOS4xMjguMSIsCiAgICAicnR0IjogMTEuNDU2LAogICAgImRlbHRhX3JldHJhbnMiOiAxLAogICAgInRvdGFsX3JldHJhbnMiOiAxCiAgfQp9Thanks, I've submitted a support ticket to our CDN provider.
di
commented
5 years ago @Juanlu001 Can you run the following command and share the results?
curl -L --output /dev/null --silent --show-error --write-out 'lookup: %{time_namelookup}\nconnect: %{time_connect}\nappconnect: %{time_appconnect}\npretransfer: %{time_pretransfer}\nredirect: %{time_redirect}\nstarttransfer: %{time_starttransfer}\ntotal: %{time_total}\n' 'https://files.pythonhosted.org/packages/6a/d1/e0d142ce7b8a5c76adbfad01d853bca84c7c0240e35577498e20bc2ade7d/virtualenv-16.2.0-py2.py3-none-any.whl'juanlu@centauri ~ $ curl -L --output /dev/null --silent --show-error --write-out 'lookup: %{time_namelookup}\nconnect: %{time_connect}\nappconnect: %{time_appconnect}\npretransfer: %{time_pretransfer}\nredirect: %{time_redirect}\nstarttransfer: %{time_starttransfer}\ntotal: %{time_total}\n' 'https://files.pythonhosted.org/packages/6a/d1/e0d142ce7b8a5c76adbfad01d853bca84c7c0240e35577498e20bc2ade7d/virtualenv-16.2.0-py2.py3-none-any.whl'
lookup: 0,028722
connect: 0,039242
appconnect: 0,087061
pretransfer: 0,087273
redirect: 0,000000
starttransfer: 60,480165
total: 61,096943
juanlu@centauri ~ $ curl -L --output /dev/null --silent --show-error --write-out 'lookup: %{time_namelookup}\nconnect: %{time_connect}\nappconnect: %{time_appconnect}\npretransfer: %{time_pretransfer}\nredirect: %{time_redirect}\nstarttransfer: %{time_starttransfer}\ntotal: %{time_total}\n' 'https://files.pythonhosted.org/packages/2d/60/f58d9e8197f911f9405bf7e02227b43a2acc2c2f1a8cbb1be5ecf6bfd0b8/pluggy-0.8.1-py2.py3-none-any.whl'
lookup: 0,028670
connect: 0,038857
appconnect: 0,087159
pretransfer: 0,087401
redirect: 0,000000
starttransfer: 0,098525
total: 0,110404Thanks @Juanlu001, can you verify if it's intermittent or occurs 100% of the time for that file? If it's only intermittent, how often does it occur? Also, could provide the full response headers when it does happen? (via curl --verbose).
astrojuanlu
commented
5 years ago Thanks @di and sorry for making you back-and-forth between me and PyPI's CDN, I appreciate the help a lot.
It happens 100 % of the time with one package and 0 % of the time with another package, and sadly curl --verbose does not give much more information:
juanlu@centauri ~ $ curl -L --output /dev/null --silent --show-error --write-out 'lookup: %{time_namelookup}\nconnect: %{time_connect}\nappconnect: %{time_appconnect}\npretransfer: %{time_pretransfer}\nredirect: %{time_redirect}\nstarttransfer: %{time_starttransfer}\ntotal: %{time_total}\n' 'https://files.pythonhosted.org/packages/6a/d1/e0d142ce7b8a5c76adbfad01d853bca84c7c0240e35577498e20bc2ade7d/virtualenv-16.2.0-py2.py3-none-any.whl' -vvv
* Trying 151.101.133.63...
* TCP_NODELAY set
* Connected to files.pythonhosted.org (151.101.133.63) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: /etc/ssl/certs
} [5 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
} [220 bytes data]
* TLSv1.2 (IN), TLS handshake, Server hello (2):
{ [106 bytes data]
* TLSv1.2 (IN), TLS handshake, Certificate (11):
{ [4222 bytes data]
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
{ [300 bytes data]
* TLSv1.2 (IN), TLS handshake, Server finished (14):
{ [4 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
} [37 bytes data]
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
} [1 bytes data]
* TLSv1.2 (OUT), TLS handshake, Finished (20):
} [16 bytes data]
* TLSv1.2 (IN), TLS handshake, Finished (20):
{ [16 bytes data]
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
* subject: C=US; ST=California; L=San Francisco; O=Fastly, Inc; CN=r.ssl.fastly.net
* start date: Sep 4 19:42:10 2018 GMT
* expire date: Apr 14 16:28:35 2019 GMT
* subjectAltName: host "files.pythonhosted.org" matched cert's "*.pythonhosted.org"
* issuer: C=BE; O=GlobalSign nv-sa; CN=GlobalSign CloudSSL CA - SHA256 - G3
* SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
} [5 bytes data]
* Using Stream ID: 1 (easy handle 0x55c225701900)
} [5 bytes data]
> GET /packages/6a/d1/e0d142ce7b8a5c76adbfad01d853bca84c7c0240e35577498e20bc2ade7d/virtualenv-16.2.0-py2.py3-none-any.whl HTTP/2
> Host: files.pythonhosted.org
> User-Agent: curl/7.58.0
> Accept: */*
>
{ [5 bytes data]
* Connection state changed (MAX_CONCURRENT_STREAMS updated)!
} [5 bytes data]
[VEEEEERY LONG DELAY]
< HTTP/2 200
< x-amz-id-2: QFAHsSLUJo9svc1jiWJXJoL/GflV/D5By2TsFp61M73GbwgiXg/fe41qZvLGH996e2oX0ba003A=
< x-amz-request-id: EBCEA7ABE1DC2518
< last-modified: Mon, 31 Dec 2018 22:16:07 GMT
< etag: "24e0adea5a7d49a25a9c9b85d70dca8c"
< x-amz-version-id: q_34KGNVGgcIMyN9twZwMSjaznKFoxWG
< content-type: binary/octet-stream
< server: AmazonS3
< cache-control: max-age=365000000, immutable, public
< accept-ranges: bytes
< date: Fri, 18 Jan 2019 18:03:17 GMT
< age: 1540029
< x-served-by: cache-sea1028-SEA, cache-mad9422-MAD
< x-cache: HIT, HIT
< x-cache-hits: 7, 3
< x-timer: S1547834598.990752,VS0,VE0
< strict-transport-security: max-age=31536000; includeSubDomains; preload
< x-frame-options: deny
< x-xss-protection: 1; mode=block
< x-content-type-options: nosniff
< x-permitted-cross-domain-policies: none
< x-robots-header: noindex
< content-length: 1921889
<
{ [2206 bytes data]
* Connection #0 to host files.pythonhosted.org left intact
lookup: 0,060748
connect: 0,071801
appconnect: 0,131620
pretransfer: 0,131986
redirect: 0,000000
starttransfer: 123,036330
total: 123,868335
juanlu@centauri ~ $
juanlu@centauri ~ $
juanlu@centauri ~ $ [VEEEEERY LONG DELAY]
[VEEEEERY: command not found
juanlu@centauri ~ $
juanlu@centauri ~ $ #curl -L --output /dev/null --silent --show-error --write-out 'lookup: %{time_namelookup}\nconnect: %{time_connect}\nappconnect: %{time_appconnect}\npretransfer: %{time_pretransfer}\nredirect: %{time_redirect}\nstarttransfer: %{time_starttransfer}\ntotal: %{time_total}\n' 'https://files.pythonhosted.org/packages/2d/60/f58d9e8197f911f9405bf7e02227b43a2acc2c2f1a8cbb1be5ecf6bfd0b8/pluggy-0.8.1-py2.py3-none-any.whl'
juanlu@centauri ~ $ curl -L --output /dev/null --silent --show-error --write-out 'lookup: %{time_namelookup}\nconnect: %{time_connect}\nappconnect: %{time_appconnect}\npretransfer: %{time_pretransfer}\nredirect: %{time_redirect}\nstarttransfer: %{time_starttransfer}\ntotal: %{time_total}\n' 'https://files.pythonhosted.org/packages/2d/60/f58d9e8197f911f9405bf7e02227b43a2acc2c2f1a8cbb1be5ecf6bfd0b8/pluggy-0.8.1-py2.py3-none-any.whl' -vvvvvv
* Trying 151.101.133.63...
* TCP_NODELAY set
* Connected to files.pythonhosted.org (151.101.133.63) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: /etc/ssl/certs
} [5 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
} [220 bytes data]
* TLSv1.2 (IN), TLS handshake, Server hello (2):
{ [106 bytes data]
* TLSv1.2 (IN), TLS handshake, Certificate (11):
{ [4222 bytes data]
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
{ [300 bytes data]
* TLSv1.2 (IN), TLS handshake, Server finished (14):
{ [4 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
} [37 bytes data]
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
} [1 bytes data]
* TLSv1.2 (OUT), TLS handshake, Finished (20):
} [16 bytes data]
* TLSv1.2 (IN), TLS handshake, Finished (20):
{ [16 bytes data]
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
* subject: C=US; ST=California; L=San Francisco; O=Fastly, Inc; CN=r.ssl.fastly.net
* start date: Sep 4 19:42:10 2018 GMT
* expire date: Apr 14 16:28:35 2019 GMT
* subjectAltName: host "files.pythonhosted.org" matched cert's "*.pythonhosted.org"
* issuer: C=BE; O=GlobalSign nv-sa; CN=GlobalSign CloudSSL CA - SHA256 - G3
* SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
} [5 bytes data]
* Using Stream ID: 1 (easy handle 0x55557562f900)
} [5 bytes data]
> GET /packages/2d/60/f58d9e8197f911f9405bf7e02227b43a2acc2c2f1a8cbb1be5ecf6bfd0b8/pluggy-0.8.1-py2.py3-none-any.whl HTTP/2
> Host: files.pythonhosted.org
> User-Agent: curl/7.58.0
> Accept: */*
>
{ [5 bytes data]
* Connection state changed (MAX_CONCURRENT_STREAMS updated)!
} [5 bytes data]
< HTTP/2 200
< x-amz-id-2: tV7rKA3ghy16z0S993tEMuBAcXMtns/yrGdf7hkAjdcHk3Q3PLqx2RsvDMrDUQrxpRlhSfhEQjc=
< x-amz-request-id: 0BBDD6FE28BCFA70
< last-modified: Wed, 09 Jan 2019 20:53:29 GMT
< etag: "2b2a30da7a38faa1c25b6efe87ccda6d"
< x-amz-version-id: 1j.JyR6Z_pD3.Rw3yLy55llF7fFJoDJt
< content-type: binary/octet-stream
< server: AmazonS3
< cache-control: max-age=365000000, immutable, public
< accept-ranges: bytes
< date: Fri, 18 Jan 2019 18:05:55 GMT
< age: 767545
< x-served-by: cache-sea1033-SEA, cache-mad9432-MAD
< x-cache: HIT, HIT
< x-cache-hits: 6, 9
< x-timer: S1547834756.790320,VS0,VE0
< strict-transport-security: max-age=31536000; includeSubDomains; preload
< x-frame-options: deny
< x-xss-protection: 1; mode=block
< x-content-type-options: nosniff
< x-permitted-cross-domain-policies: none
< x-robots-header: noindex
< content-length: 17274
<
{ [2208 bytes data]
* Connection #0 to host files.pythonhosted.org left intact
lookup: 0,004186
connect: 0,014426
appconnect: 0,048488
pretransfer: 0,048716
redirect: 0,000000
starttransfer: 0,059478
total: 0,071606Hi @di! I guess there hasn't been much progress with this issue, but did the CDN say anything that I can use to try to fix this with my ISP at least?
di
commented
5 years ago Hi @Juanlu001, sorry for the delay. Our provider wants to confirm that you're seeing this issue on the WAN side of your router, and if so, if you can send then a packet capture they can inspect.
(If that's not possible or you're not sure how to do that, it's no problem, just let me know)
astrojuanlu
commented
5 years ago I don't know what WAN is, so if there are some instructions I can follow, I'll generate a packet capture.
supine
commented
5 years ago @Juanlu001 by WAN we mean the ISP side of your router.
Not sure what kind of connection it is but is it possible to connect a computer directly to it rather than using your router?
If so, please capture the slow and normal downloads in that configuration.
If not, please just take captures of the slow and normal downloads in your standard configuration and we will see if we can extract any insight from them.
astrojuanlu
commented
5 years ago Sorry for the lack of response. I have observed that it also happens with other websites, and apparently is not as predictable as I thought in the beginning, so I'm closing this issue and I will try to contact my ISP directly when I have more information. Thanks for lending a hand :pray:
My Platform
I noticed while doing
pip install toxon a Python 3.4 box that some dependencies were taking a very long time to download (sometimes 2 minutes), longer than the default timeout time and therefore requiring manual environment configuration. It is not a pip problem though:First attempt
From 18:31:37 to 18:33:29, data download in 0.6 seconds.
Second attempt
From 18:34:30 to 18:36:22, data download in 0.5 s.
Other packages
Total download time less than 1 second.
Network telemetry
DNS Resolution
Traceroutes
IPv4
HTTPS Requests
IPv4
TLS Debug
IPv4