Closed cpauya closed 9 years ago
Experienced this again @aronasorman
Hi @cpauya , I also experience the same issue.
I used a VPN and re-tried the download but got the same issue after downloading about 117MB+.
Just want to mention that this also happens on the Bamboo (dungeon) server.
I notice that downloading the assessment.zip, it always connection closed
What's up with all the screenshots, can't you Mac guys copy paste text from your terminals!? :D
This issue will also affect the Debian and PyPi installation methods. Therefore, I suggest we find the real problem, what causes the connection to break.
I use wget, which I think is also available on Mac, and it looks like what you're using. It has settings to resume broken downloads and retry automatically. See:
http://superuser.com/questions/493640/how-to-retry-connections-with-wget
In all circumstances, we should try a more robust wget command. And figure out what's up with the central server.
I think the issue was missed in yesterday's 0.15 release meeting because it's a release blocker not in the main repo. @MCGallaspy @aronasorman take note :)
Oh yes we can too, just preferred to use a screenshot to include more info on the screen (like the URL of the browser)... we'll do better to include text for "search purposes" too. :smile_cat:
We do have the wget
command, but curl
can also retry broken downloads as per this table http://curl.haxx.se/docs/comparison-table.html. Sorry, I think we missed to use that option on our setup script. @arceduardvincent let's use the --retry 20 :smile_cat:
But I agree that something is up with the server - and has only experienced this issue the past few weeks.
Tried it with curl --retry 20 ...
and still got this:
(ka-lite-installers)MacBook-Air:osx cyril$ ./setup.sh "https://github.com/learningequality/ka-lite/archive/0.15.x.zip" "https://learningequality.org/downloads/ka-lite/0.15/content/khan_assessment.zip"
Using temporary directory /Users/cyril/Documents/work/installers/osx/temp...
2/13. Downloading assessment
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 468M 0 81662 0 0 883 0 6d 10h 0:01:32 6d 10h 0curl: (18) transfer closed with 490838977 bytes remaining to read
./setup.sh: Can't download 'https://learningequality.org/downloads/ka-lite/0.15/content/khan_assessment.zip', exiting...
Sorry I don't know Curl, I linked to a guide for wget that shows how you can download and automatically retry on connection errors. The --retry
option for curl may not be the same.
Either try the man page or look for similar articles.
Used wget
as per your suggestion @benjaoming - and here's the output for "try:19 and try:20":
wget --retry-connrefused --read-timeout=20 --waitretry=1 --continue -O $ASSESSMENT_PATH $ASSESSMENT_URL
...
--2015-10-07 00:41:51-- (try:19) https://learningequality.org/downloads/ka-lite/0.15/content/khan_assessment.zip
Connecting to learningequality.org|23.253.32.238|:443... connected.
HTTP request sent, awaiting response... 206 Partial Content
Length: 490920639 (468M), 452915115 (432M) remaining [application/zip]
Saving to: '/Users/cyril/Documents/work/installers/osx/temp/assessment.zip'
/Users/cyril/Documents/work/installer 7%[+++++ ] 36.33M --.-KB/s in 20s m
2015-10-07 00:42:13 (4.14 KB/s) - Read error at byte 38091240/490920639 (Operation timed out). Retrying.
--2015-10-07 00:42:14-- (try:20) https://learningequality.org/downloads/ka-lite/0.15/content/khan_assessment.zip
Connecting to learningequality.org|23.253.32.238|:443... connected.
HTTP request sent, awaiting response... 206 Partial Content
Length: 490920639 (468M), 452829399 (432M) remaining [application/zip]
Saving to: '/Users/cyril/Documents/work/installers/osx/temp/assessment.zip'
/Users/cyril/Documents/work/installer 9%[+++++> ] 42.72M --.-KB/s in 28s s
2015-10-07 00:42:43 (237 KB/s) - Read error at byte 44796092/490920639 (Operation timed out). Giving up.
./setup.sh: Can't download 'https://learningequality.org/downloads/ka-lite/0.15/content/khan_assessment.zip', exiting...
(ka-lite-installers)MacBook-Air:osx cyril$
Perhaps setting the number of retries option to -t 100
would suffice?
This is great, looks like we now have some semi-reliable way of replicating this error.
@cpauya it would be sane to make a couple of retries in case the client side has a bad wireless or something. Trying it 100 times is maybe a tad too much, especially if there's no connection at all, and we have to wait for it 100 times before failing. Maybe trying 5 times is enough.
@aronasorman have a look at the servers socket timeouts, it's definitely an nginx setting.
I've increased the server timeout. Please post here if you keep getting this regularly.
@benjaoming, as you already know, the default retry value of wget
is 20, that's what we got as per above. Using ratio and proportion for 20 retries: 9% competion, it'll take about 222.22 retries to complete the 100% download. :smile_cat:
For a practical application in our country, the retry value of 20 will not be enough for us. We can run the script and wait for it to finish and work on other tasks while it's trying to complete it's download.
Note: I also experienced this on the Bamboo server having reached about 18 retries before completing the download.
We can cancel the script anytime anyways, so I think let's try with 100 for now and lower it when we don't encounter issues.
@aronasorman - Tried this today and finished download on retry 57 for about 30 minutes.
(ka-lite-installers)MacBook-Air:osx cyril$ ./setup.sh "https://github.com/learningequality/ka-lite/archive/0.15.x.zip" "https://learningequality.org/downloads/ka-lite/0.15/content/khan_assessment.zip"
Using temporary directory /Users/cyril/Documents/work/installers/osx/temp...
2/13. Downloading assessment
--2015-10-07 10:20:03-- https://learningequality.org/downloads/ka-lite/0.15/content/khan_assessment.zip
Resolving learningequality.org... 23.253.32.238
Connecting to learningequality.org|23.253.32.238|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 490920639 (468M) [application/zip]
Saving to: '/Users/cyril/Documents/work/installers/osx/temp/assessment.zip'
/Users/cyril/Documents/work/installe 0%[ ] 79.75K --.-KB/s in 20s m
2015-10-07 10:20:26 (3.94 KB/s) - Read error at byte 81662/490920639 (Operation timed out). Retrying.
...
...
...
--2015-10-07 10:49:05-- (try:57) https://learningequality.org/downloads/ka-lite/0.15/content/khan_assessment.zip
Connecting to learningequality.org|23.253.32.238|:443... connected.
HTTP request sent, awaiting response... 206 Partial Content
Length: 490920639 (468M), 119389177 (114M) remaining [application/zip]
Saving to: '/Users/cyril/Documents/work/installers/osx/temp/assessment.zip'
/Users/cyril/Documents/work/installe 100%[+++++++++++++++++++++++++++++++++++++++++++++++++++++++=================>] 468.18M 919KB/s in 2m 50s
2015-10-07 10:51:57 (686 KB/s) - '/Users/cyril/Documents/work/installers/osx/temp/assessment.zip' saved [490920639/490920639]
@cpauya please continue the good research :) I will copy it for the debian scripts :)
@aronasorman any news here?
Should be solved now ever since this was implemented on the gateway server: https://github.com/fle-internal/ansible-playbooks/commit/727925ab407aba70eac590d295cc95a83bff97a3
@aronasorman which is the most permanant link for assessment items? pantry or leq.org ? Is Pantry something we might rename and therefore it's better to use learningequality.org straight?
better to use leq.org/downloads, since we get stats from there.
Branch develop
Target 0.15.x
Expected Behavior Download of assessment.zip should succeed.
Current Behavior Downloading assessment.zip sometimes does not succeed and is closed by remote server.
Related Issues
Screenshot