search

jaryaman / propNews

Making the news proportionate to global priorities
GNU Affero General Public License v3.0
0 stars 0 forks source link

Investigate the optimal call frequency of NewsAPI #19

Open jaryaman opened 5 years ago

jaryaman commented 5 years ago

We are limited by page_limit_per_request=10 and results_per_page=100. Is it possible that, as we increase the list of news sources we search, we hit the limit that NewsAPI can return in a single call, and we therefore need to query NewsAPI more frequently?

jaryaman commented 5 years ago

In fact, we may want to make news_refresh_period take some maximum value which we infer, to prevent the database developing time gaps, and throw exceptions if NewsAPI doesn't allow us to query frequently enough. Not sure if this is possible at this point since we depend on NewsAPI to find when articles are published. Unclear if this issue is a bug, or an enhancement request.