Downloads from Sourceforge failing with "make gcw0_defconfig"

[Drakonas]

I end up having to manually download each Sourceforge tar archive, because the script no longer pulls the archive properly (downloads the webpage instead of the file is my guess).

Output error:

>>> p7zip 9.20.1 Downloading
--2015-04-19 21:36:18--  http://downloads.sourceforge.net/project/p7zip/p7zip/9.20.1/p7zip_9.20.1_src_all.tar.bz2
Resolving downloads.sourceforge.net (downloads.sourceforge.net)... 216.34.181.59
Connecting to downloads.sourceforge.net (downloads.sourceforge.net)|216.34.181.59|:80...  connected.
HTTP request sent, awaiting response... 307 Temporary Redirect
Location: http://sourceforge.net/projects/p7zip/files/p7zip/9.20.1/p7zip_9.20.1_src_all.tar.bz2/download?use_mirror=colocrossing [following]
--2015-04-19 21:36:18--  http://sourceforge.net/projects/p7zip/files/p7zip/9.20.1/p7zip_9.20.1_src_all.tar.bz2/download?use_mirror=colocrossing
Resolving sourceforge.net (sourceforge.net)... 216.34.181.60
Connecting to sourceforge.net (sourceforge.net)|216.34.181.60|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 14580 (14K) [text/html]
Saving to: ‘/home/larry/Downloads/NathanStuff/N64/buildroot/dl/p7zip_9.20.1_src_all.tar.bz2.tmp’

100%[======================================>] 14,580      --.-K/s   in 0.04s

2015-04-19 21:36:19 (335 KB/s) - ‘/home/larry/Downloads/NathanStuff/N64/buildroot/dl/p7zip_9.20.1_src_all.tar.bz2.tmp’ saved [14580/14580]

>>> p7zip 9.20.1 Extracting
bzcat /home/larry/Downloads/NathanStuff/N64/buildroot/dl/p7zip_9.20.1_src_all.tar.bz2 | tar --strip-components=1 -C /home/larry/Downloads/NathanStuff/N64/buildroot/output/build/p7zip-9.20.1  -xf -
bzcat: /home/larry/Downloads/NathanStuff/N64/buildroot/dl/p7zip_9.20.1_src_all.tar.bz2 is not a bzip2 file.
tar: This does not look like a tar archive
tar: Exiting with failure status due to previous errors
make: *** [/home/larry/Downloads/NathanStuff/N64/buildroot/output/build/p7zip-9.20.1/.stamp_extracted] Error 2

There have been 100's of these, and I am having to redownload every single one... *sigh

[mthuurne]

I can't reproduce this problem:

>>> p7zip 9.20.1 Downloading
--2015-04-20 13:27:22--  http://downloads.sourceforge.net/project/p7zip/p7zip/9.20.1/p7zip_9.20.1_src_all.tar.bz2
Resolving downloads.sourceforge.net (downloads.sourceforge.net)... 216.34.181.59
Connecting to downloads.sourceforge.net (downloads.sourceforge.net)|216.34.181.59|:80... connected.
HTTP request sent, awaiting response... 302 Found
Location: http://heanet.dl.sourceforge.net/project/p7zip/p7zip/9.20.1/p7zip_9.20.1_src_all.tar.bz2 [following]
--2015-04-20 13:27:22--  http://heanet.dl.sourceforge.net/project/p7zip/p7zip/9.20.1/p7zip_9.20.1_src_all.tar.bz2
Resolving heanet.dl.sourceforge.net (heanet.dl.sourceforge.net)... 193.1.193.66, 2001:770:18:aa40::c101:c142
Connecting to heanet.dl.sourceforge.net (heanet.dl.sourceforge.net)|193.1.193.66|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 3835235 (3.7M) [application/octet-stream]
Saving to: ‘/home/mth/gcw0/buildroot/output/build/.p7zip_9.20.1_src_all.tar.bz2.212RwI/output’

/home/mth/gcw0/buildroot/ 100%[==================================================>]   3.66M  3.10MB/s   in 1.2s   

2015-04-20 13:27:24 (3.10 MB/s) - ‘/home/mth/gcw0/buildroot/output/build/.p7zip_9.20.1_src_all.tar.bz2.212RwI/output’ saved [3835235/3835235]

Is it possible it was a temporary problem at SourceForge that has since been fixed?

Which version of wget are you using? (1.16 here) Is there any environment variable or wgetrc that might alter its behavior?

[Drakonas]

Odd.. I'm using Ubuntu 14.04.1, and there isn't an update for wget to 1.16. And no, I checked for any Bash aliases, and a ~/.wgetrc does not exist. Nothing is modifying the behavior.

The build finally finished after downloading each zip separately btw, and I was able to build the kernel and rootfs. The rootfs updated properly on my GCW-Zero, but the kernel build caused a bootloop. However, that is another bug to discuss.

EDIT: I think the key problem is Sourceforge is redirecting me to a mirror each time.

Notice in mine, the URL redirects to http://sourceforge.net/projects/p7zip/files/p7zip/9.20.1/p7zip_9.20.1_src_all.tar.bz2/download?use_mirror=colocrossing

Try clicking that link. You can see the redirect behavior in the browser.

In your example, it does not have this mirror redirect in the received URL. This must depend on location. Maybe there's a setting on Sourceforge's website to change the mirror.

UPDATE: When I attempt wget without the makefile's environment variable setup, the exact same download URL that you used works fine for me when run myself: wget -O p7zip_9.20.1_src_all.tar.bz2 'http://downloads.sourceforge.net/project/p7zip/p7zip/9.20.1/p7zip_9.20.1_src_all.tar.bz2'

Mayhaps it was a temporary issue. However, as per a guide found here, I think your URLs need updating (unfortunately -.-), you might not be using the best URL format for wget'ing Sourceforge files. From my experience, it is not recommended to copy the direct download link on the 5-second download timer page, but rather copy the file link from the file list, and let wget properly get the redirect itself. Perhaps this is recommended because of mirror differences?

For example, the URL you used for p7zip was this: http://downloads.sourceforge.net/project/p7zip/p7zip/9.20.1/p7zip_9.20.1_src_all.tar.bz2 It is recommended to use this instead: http://sourceforge.net/projects/p7zip/files/p7zip/9.20.1/p7zip_9.20.1_src_all.tar.bz2/download

The only problem is that this would be a massive undertaking to fix this. -.- And I realize it.