search

ChristianSch / crime-stats-nlp

NLP based crime stats extraction for police reports taken from http://www.presseportal.de
MIT License
2 stars 0 forks source link

multi threaded urlopen #3

Open ChristianSch opened 9 years ago

ChristianSch commented 9 years ago

in lib.py we should utilise threads to retrieve:

  1. article urls from pagination (1 thread per pagination url)
  2. article text (a fixed amount of threads; more of a thread pool)