alexgand / springer_free_books

Python script to download all Springer books released for free during the 2020 COVID-19 quarantine
GNU General Public License v3.0
1.64k stars 366 forks source link

memory error using 32 bit python in windows #29

Closed kodabey closed 4 years ago

kodabey commented 4 years ago

If you install the 32 bit version of Python, this script will throw an error on fast internet connections, since the memory buffer size shoots over what 32 bit python can handle. Installing the 64 bit version mitigates the issue.

humanfactors commented 4 years ago

I'm also getting constant memory errors in 64bit python.

MemoryError                               Traceback (most recent call last)
<ipython-input-4-beb7b710717e> in <module>
     35 
     36     if not os.path.exists(output_file):
---> 37         myfile = requests.get(new_url, allow_redirects=True)
     38         try:
     39             open(output_file, 'wb').write(myfile.content)

c:\program files (x86)\python36-32\lib\site-packages\requests\api.py in get(url, params, **kwargs)
     74 
     75     kwargs.setdefault('allow_redirects', True)
---> 76     return request('get', url, params=params, **kwargs)
     77 
     78 

c:\program files (x86)\python36-32\lib\site-packages\requests\api.py in request(method, url, **kwargs)
     59     # cases, and look like a memory leak in others.
     60     with sessions.Session() as session:
---> 61         return session.request(method=method, url=url, **kwargs)
     62 
     63 

c:\program files (x86)\python36-32\lib\site-packages\requests\sessions.py in request(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redirects, proxies, hooks, stream, verify, cert, json)
    528         }
    529         send_kwargs.update(settings)
--> 530         resp = self.send(prep, **send_kwargs)
    531 
    532         return resp

c:\program files (x86)\python36-32\lib\site-packages\requests\sessions.py in send(self, request, **kwargs)
    681 
    682         if not stream:
--> 683             r.content
    684 
    685         return r

c:\program files (x86)\python36-32\lib\site-packages\requests\models.py in content(self)
    827                 self._content = None
    828             else:
--> 829                 self._content = b''.join(self.iter_content(CONTENT_CHUNK_SIZE)) or b''
    830 
    831         self._content_consumed = True

MemoryError: 

I've had to reset the download start point at least 15 - 20 times....

Update: I'm a fatigued derp today thanks team

alexgand commented 4 years ago

I'm also getting constant memory errors in 64bit python.

Strange. How much RAM do you have in your system?

jaintj95 commented 4 years ago

@humanfactors From the logs you shared, your python version is clearly 32 bit. If you have a 64bit system, you need to uninstall python and reinstall the 64 bit package

kodabey commented 4 years ago

This might not be the right place to be post this but in the interest of helping newer coders...the 64 bit version of Microsoft Windows Python doesn't update your path variable correctly on install so you have to do that manually for the windows instructions on the main page of this script to work.

humanfactors commented 4 years ago

Thanks all, to be honest I'm surprised I even have a 32bit install on here. Thanks for the help @kodabey 😀