search

AndyTheFactory / newspaper4k

📰 Newspaper4k a fork of the beloved Newspaper3k. Extraction of articles, titles, and metadata from news websites.
MIT License
429 stars 37 forks source link

Obtaining -new- news each day #206

Open AndyTheFactory opened 10 months ago

AndyTheFactory commented 10 months ago

Issue by durakkerem Tue May 8 20:34:27 2018 Originally opened as https://github.com/codelucas/newspaper/issues/563

So I know that I can building a news site crawls over all available news of the website:

cnn_paper = newspaper.build('https://cnn.com')

But how about when I want to get only newest news? In my case default caching architecture of Newspaper3k does not work because I run a never-ending crawler and my cache will finally end up overflowing.

One other problem is I can obtain publish_date of articles only after downloading&parsing each news.

Is there a workaround for this case? I think it will be very useful for many users too.

AndyTheFactory commented 10 months ago

Comment by jonkislin Mon Apr 29 16:01:43 2019

@codelucas I have the same issue and I'm also interested in this functionality; Is there something @durakkerem and I may be missing?

Awesome library btw!