amnong
commented
11 years ago Well, I do agree it's wrong to use response.raw, I genuinely don't remember why I did that, but I do remember avoiding response.content for some reason, maybe because I thought or suspected it buffers the entire response body, I don't really remember. In any case I'm pretty swamped at work right now, and kinda lazy on the weekends :) If you happen to find a quick solution I'd appreciate a patch, and if you don't then I'll try to get around to testing and fixing this some time soon.
BTW it seems to me the potential here is for downloading an entirely corrupt file.
In the download function, it's currently set to use response.raw, via shutil.copyfileobj. It's easy to see that this avoids loading the whole downloaded file into memory, as opposed to the commented out line next to it that just writes response.content.
However, if I'm reading the 'requests' documentation correctly, response.content will automatically handle gzip/deflate and result in the correct file if the server uses these, while response.raw, being genuinely raw socket data, will not handle this, and the resulting file will not be what is expected.
Am I right or not? I don't currently have a compliant server to test that against.