search

easlice / bandcamp-downloader

Download your bandcamp collection using this python script.
MIT License
281 stars 34 forks source link

Use of requests library could lead to partial file downloads without notice #10

Closed puigru closed 1 year ago

puigru commented 1 year ago

I've recently begun using this tool to build my music library. However, I noticed something odd after running it a second time, as it re-downloaded a couple of files because their size did not match up with what it should have been. Yet, there had been no errors on the previous run. I quickly realized this could be another instance of a well-known requests shortcoming.

Unfortunately, unlike other HTTP tools, requests does not check the response's Content-Length header to ensure it has been fully downloaded, leaving the burden on its user to do so. Hence, it's possible to end up with partial downloads by using bandcamp-downloader, as it does not perform this check. Although, it will attempt to re-download these files on subsequent runs.

Read: https://blog.petrzemek.net/2018/04/22/on-incomplete-http-reads-and-the-requests-library-in-python/ https://github.com/psf/requests/issues/4956 (and issues linked within)

easlice commented 1 year ago

I thought I had seen this before, but I wasn't sure as to the cause. I appreciate the effort and the input of someone who knows these libs better than I do! Also, I really appreciate your thorough explanations and submitting patches to boot!