StegSchreck / RatS

Movie Ratings Synchronization with Python
GNU Affero General Public License v3.0
272 stars 31 forks source link

Re-Implement Sites using the official APIs #306

Open StegSchreck opened 7 months ago

StegSchreck commented 7 months ago

Scraping / Inserting data via Selenium was originally done to make the usage of this synchronisation script easier. Especially since some sites didn't offer an API, it was not free to use, or it was lacking data (e.g. for private profiles).

Since some sites have introduced some quite effective anti-bot measures (e.g. captchas on IMDb), this reduces the usefulness of this script/application.

With this issue, re-implement the most-important APIs to replace the Selenium-based scraping&inserting. The sites where a simple CSV upload/download is performed are out-of scope for the first phase of this.

The user might then have to generate private API credentials (e.g. client ID + secret) - this would need to be documented as a step-by-step how-to guide.

For each of the sites, this should be a feature toggle to let the user chose which way to use.

StegSchreck commented 7 months ago

Official API documentations

Potential 3rd party libraries

to be evaluated for being uptodate, feature-complete, etc. and possible alternatives

No known API available