cavaliergopher / grab

A download manager package for Go
BSD 3-Clause "New" or "Revised" License
1.38k stars 151 forks source link

Multiple downloads to the same file result in bad content length #61

Closed yene closed 5 years ago

yene commented 5 years ago

https://gist.github.com/yene/1f4f7da1ba2955590246480e8a17d802

$ go run grab-bug.go 
2019/04/27 17:42:40 finished download
2019/04/27 17:42:40 Could not download https://trello-attachments.s3.amazonaws.com/52460829b7e0c8990b00008e/59665dfd12eb088169fe65a0/3c1ea606328206cdaa4779183fa60bdb/image.png
2019/04/27 17:42:40 bad content length

Found this by downloading multiple files to the same filename, it seems statFileInfo has the filesize of the previous file.

cavaliercoder commented 5 years ago

This is correct behavior by design. In your gist, the name of each file will be guessed by grab to be ./image.png and will be downloaded to the same local path: ./image.png. So each download attempt to resume the previous and results in corruption.

You will need to give each download a unique local path. Maybe:

for i, url := range testURLs {
    req, _ := grab.NewRequest(fmt.Sprintf("./image_%d", i), url)
    resp := client.Do(req)
    log.Println("finished download")
    if err := resp.Err(); err != nil {
        log.Println("Could not download", url)
        log.Fatalln(err)
    }
}
yene commented 5 years ago

Yeah I did. But that it treats a previous downloaded file as resumed still caught me off guard.