search

codelucas / newspaper

newspaper3k is a news, full-text, and article metadata extraction in Python 3. Advanced docs:
https://goo.gl/VX41yK
MIT License
14.19k stars 2.12k forks source link

URL-specific memoize? #232

Open ChristopherLucas opened 8 years ago

ChristopherLucas commented 8 years ago

Is there a straightforward way to memoize by url? EG, a CNN- and NBC-specific directory memoizing these crawls?

Thanks!

yprez commented 8 years ago

@ChristopherLucas I think the best approach would be storing the parsed data outside of newspaper.. What exactly is your usecase?