search

codelucas / newspaper

newspaper3k is a news, full-text, and article metadata extraction in Python 3. Advanced docs:
https://goo.gl/VX41yK
MIT License
14.06k stars 2.11k forks source link

Article Multi-threading downloading #413

Open gumpjt opened 7 years ago

gumpjt commented 7 years ago

Hi there, In the example, I need to setup some "root url", like espn.com, slate.com, then I can use the "multi-threading" mechanism. But now I already have thousands of article links, every single link is an article page, how to use the "multi-threading" to speed up?

Thanks.

unkwn1-repo commented 5 years ago

Not sure if it's possible as there's nothing in the newspaper3k documentation that alludes to this being possible from within the library.

Instead you may have to incorporate threading manually.

There are some great guides here: Stackoverflow - Python Threading