amusecode / amuse

Astrophysical Multipurpose Software Environment. This is the main repository for AMUSE
http://www.amusecode.org
Apache License 2.0
155 stars 98 forks source link

MESA dependencies from SDK fail to download with wget #1064

Closed LourensVeen closed 1 month ago

LourensVeen commented 1 month ago

Describe the bug When installing mesa_r15140, we download various dependencies. Some of them come from http://user.astro.wisc.edu/~townsend, and that server seems to be blocking wget. I can download them just fine using curl, or when telling wget not to send a user agent.

To Reproduce Steps to reproduce the behavior:

$ wget http://user.astro.wisc.edu/~townsend/resource/download/sdk2/src/lapack95.tgz
--2024-07-29 11:16:02--  http://user.astro.wisc.edu/~townsend/resource/download/sdk2/src/lapack95.tgz
Resolving user.astro.wisc.edu (user.astro.wisc.edu)... 144.92.179.14
Connecting to user.astro.wisc.edu (user.astro.wisc.edu)|144.92.179.14|:80... connected.
HTTP request sent, awaiting response... 403 Forbidden
2024-07-29 11:16:02 ERROR 403: Forbidden.

Expected behavior A clear and concise description of what you expected to happen.

$ curl http://user.astro.wisc.edu/~townsend/resource/download/sdk2/src/lapack95.tgz >lapack95.tgz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 1542k  100 1542k    0     0   241k      0  0:00:06  0:00:06 --:--:--  325k
$ wget --user-agent="" http://user.astro.wisc.edu/~townsend/resource/download/sdk2/src/lapack95.tgz
--2024-07-29 11:20:19--  http://user.astro.wisc.edu/~townsend/resource/download/sdk2/src/lapack95.tgz
Resolving user.astro.wisc.edu (user.astro.wisc.edu)... 144.92.179.14
Connecting to user.astro.wisc.edu (user.astro.wisc.edu)|144.92.179.14|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1579613 (1.5M) [application/x-gzip]
Saving to: ‘lapack95.tgz.1’

lapack95.tgz.1                                  100%[====================================================================================================>]   1.51M   304KB/s    in 7.1s    

2024-07-29 11:20:26 (218 KB/s) - ‘lapack95.tgz.1’ saved [1579613/1579613]

Perhaps the server could be fixed? Or, if one is only supposed to access it using a web browser, curl blocked as well? :smile:

@rhdtownsend

rhdtownsend commented 1 month ago

You need to use the -user-agent=“" flag with wget — see the FAQ:

http://user.astro.wisc.edu/~townsend/static.php?ref=mesasdk

cheers,

Rich

On Jul 29, 2024, at 4:27 AM, Lourens Veen @.***> wrote:

Describe the bug When installing mesa_r15140, we download various dependencies. Some of them come from http://user.astro.wisc.edu/~townsend, and that server seems to be blocking wget. I can download them just fine using curl, or when telling wget not to send a user agent. To Reproduce Steps to reproduce the behavior: $ wget http://user.astro.wisc.edu/~townsend/resource/download/sdk2/src/lapack95.tgz --2024-07-29 11:16:02-- http://user.astro.wisc.edu/~townsend/resource/download/sdk2/src/lapack95.tgz Resolving user.astro.wisc.edu (user.astro.wisc.edu)... 144.92.179.14 Connecting to user.astro.wisc.edu (user.astro.wisc.edu)|144.92.179.14|:80... connected. HTTP request sent, awaiting response... 403 Forbidden 2024-07-29 11:16:02 ERROR 403: Forbidden.

Expected behavior A clear and concise description of what you expected to happen. $ curl http://user.astro.wisc.edu/~townsend/resource/download/sdk2/src/lapack95.tgz >lapack95.tgz % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 1542k 100 1542k 0 0 241k 0 0:00:06 0:00:06 --:--:-- 325k

$ wget --user-agent="" http://user.astro.wisc.edu/~townsend/resource/download/sdk2/src/lapack95.tgz --2024-07-29 11:20:19-- http://user.astro.wisc.edu/~townsend/resource/download/sdk2/src/lapack95.tgz Resolving user.astro.wisc.edu (user.astro.wisc.edu)... 144.92.179.14 Connecting to user.astro.wisc.edu (user.astro.wisc.edu)|144.92.179.14|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 1579613 (1.5M) [application/x-gzip] Saving to: ‘lapack95.tgz.1’

lapack95.tgz.1 100%[====================================================================================================>] 1.51M 304KB/s in 7.1s

2024-07-29 11:20:26 (218 KB/s) - ‘lapack95.tgz.1’ saved [1579613/1579613]

Perhaps the server could be fixed? Or, if one is only supposed to access it using a web browser, curl blocked as well? 😄 @rhdtownsend — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

-- Rich Townsend • Professor of Astronomy Astronomy Department • University of Wisconsin-Madison Phone: 608-262-1752 • E-mail: @.***

LourensVeen commented 1 month ago

Ah, sorry, it hadn't occurred to me to do a web search. I'll tell wget to sneak in under the radar, thanks!

rhdtownsend commented 1 month ago

I’d rather fix it on the webserver side, but I don’t have control over the machine! Hence the silly hack….

On Jul 29, 2024, at 10:49 AM, Lourens Veen @.***> wrote:

Ah, sorry, it hadn't occurred to me to do a web search. I'll tell wget to sneak in under the radar, thanks! — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

-- Rich Townsend • Professor of Astronomy Astronomy Department • University of Wisconsin-Madison Phone: 608-262-1752 • E-mail: @.***