jansmolders86 / mediacenterjs

A HTML/CSS/Javascript (NodeJS) based Media center
http://mediacenterjs.com
1.29k stars 243 forks source link

Prevent bad movie scraper data due to api limitations #83

Closed jansmolders86 closed 10 years ago

jansmolders86 commented 10 years ago

The current api used to scrape movie data only allows ten calls per minute. Resulting in potential misfires and wrongfully represented data when checking a large amount of files rapidly.

jansmolders86 commented 10 years ago

Currently, the scaper will fail to retrieve information from tmdb after several requests due to the limitations of the free api. When this happens, the scraper will continue to scrape the movies but place placeholder data in the database. If a user refreshes the page or the scheduled task kicks in, the scraper will try to get the real data for movies that have placeholder data. This is ofcourse a temporary fix, hopefullly there will be a paid api in the future to fix this problem completely.