search

alexgand / springer_free_books

Python script to download all Springer books released for free during the 2020 COVID-19 quarantine
GNU General Public License v3.0
1.64k stars 366 forks source link

Fixed memory error problem #35

Closed chaosAD closed 4 years ago

chaosAD commented 4 years ago

The original code keeps the download content in the memory before writing to the file. If the file is very large, then it will cause memory error in the requests.get(). I fixed it by using streaming mode, thus it resolves #33.

I added requirements2x.txt for python 2.7.x users. I know I should be upgrading to Python 3 ;-)

I also fixed filename-too-long issue by truncating it. Some files have names longer than 150 characters, thus generating IOError. It happens under Python 2.7.x. I am not sure if Python 3 has the same issue or not.

alexgand commented 4 years ago

Hi, it must have something wrong. When I run the new script, all files are corrupt and about ~10k in size. Thanks.

chaosAD commented 4 years ago

My apologies, the code was using an uninitialized url variable (a symptom of cut-and-paste from my test code). I have fixed it.

alexgand commented 4 years ago

Merged, thanks!