search

floodsung / Deep-Learning-Papers-Reading-Roadmap

Deep Learning papers reading roadmap for anyone who are eager to learn this amazing tech!
38.36k stars 7.32k forks source link

Fixed download.py (Empty file downloads, encoding errors) #65

Closed shreyaspadhy closed 7 years ago

shreyaspadhy commented 7 years ago

Fixed the following errors -

  1. Python defaulting to ASCII encoding instead of Unicode. Set global encoding to 'utf-8'
  2. Script downloading 0kb empty FILE objects. Some filenames being generated contained ":", which is a restricted character in filenames on Windows and MacOS and was truncating filenames before the extension, added to the replacements list in clean_text.
  3. Download failures. Used the requests library to improve success rates of downloads, which seemed to work a bit better.
zeyu42 commented 7 years ago

To run the script in Python 3, one has to remove the line import urllib2 and the encoding settings.

shreyaspadhy commented 7 years ago

Does Python 3 automatically encode in Unicode then? I'll make a few changes to make sure it works in Python 3, thanks

zeyu42 commented 7 years ago

Yes. BTW thanks for this wonderful repo!