search

cavaliergopher / grab

A download manager package for Go
BSD 3-Clause "New" or "Revised" License
1.38k stars 151 forks source link

Fix problem with Range bytes and multiple hosts via redirect #83

Closed ViRb3 closed 3 years ago

ViRb3 commented 3 years ago

First of all thank you so much for this amazing library!

I'm attempting to download files from a host that uses a CDN. The host only acts as a resolver, so every response will be a redirect to the appropriate CDN resource. Now, imagine a file is only partially downloaded, and grab is attempting to resume. It will send a HEAD request to the host, which will redirect to the CDN, and the CDN will say that it supports partial downloads (Range). Grab will then proceed to send a GET request with Range header to the host, which returns 404 because it does not support Range - it was the CDN that advertised support for it. The correct scenario here, in my opinion, is to remember the redirect destination during The HEAD operation, and use that for future requests. This is what this CR achieves, in 2 lines of code :)

cavaliercoder commented 3 years ago

It seems my automated GitHub tests broke some time ago. Do all tests pass for this diff?

ViRb3 commented 3 years ago

I believe they pass.

root@grab# go test ./...
ok      github.com/cavaliercoder/grab   2.554s
ok      github.com/cavaliercoder/grab/bps       0.037s
?       github.com/cavaliercoder/grab/cmd/grab  [no test files]
ok      github.com/cavaliercoder/grab/grabtest  0.180s
?       github.com/cavaliercoder/grab/grabui    [no test files]
cavaliercoder commented 3 years ago

Love it. Thank you for submitting this diff!