[Suggestion] Use the official iMDB datasets that are more updated than OMDB. - Githubissues

mynttt / UpdateTool

A tool to update the IMDB ratings for Plex libraries that contain movies/series and use the IMDB agent to receive ratings

GNU General Public License v3.0

250 stars 12 forks source link

[Suggestion] Use the official iMDB datasets that are more updated than OMDB. #11

Closed KurtzPT closed 4 years ago

KurtzPT commented 4 years ago

Using the official datasets of IMDB (https://datasets.imdbws.com/) only has advantages:

Updated every day (Not sure of this, but certainly updated way more frequently than OMDB, it's easy to check this by comparing recent movies rating in both sides)
No need to add an API key
You only need to download the title.ratings.tsv.gz each time you run the app. Every single rating in IMDB is in that file so you don't need to have multiple calls to OMDB API.

The only "cons" is that you need to implement a CSV type of parser to read the file after extracting it.

mynttt commented 4 years ago

I did look at the dataset a few weeks ago and it looks promising. Do you know how frequently it is updated? I think on Monday I can find some time to implement it, before I release it I would run it against my movie library to see how up to date it is. If it looks fine it really is a better alternative towards OMDB, too bad they don't version their datasets that would help to avoid re downloading it every time the tool is started. But it's only 4MB compressed so no big deal.

KurtzPT commented 4 years ago

Last update was today (31 Jan) at around 1 pm GMT time. My guess it's either updated daily or weekly. I'll have to check tomorrow if the datasets were updated.

mynttt commented 4 years ago

That sounds like it's a good source of data. Thanks for bringing this up here!

mynttt commented 4 years ago

Implemented in a seperate branch. I will merge it into the master when I updated the documentation so users that come to the repository won't be confused. The IMDB dataset appears to be be refreshed daily which is really nice!

mynttt commented 4 years ago

Migration has been completed. Feature is live, thanks for contributing!

ShrinkWrapper commented 4 years ago

Seems that the dataset hasn't updated in a couple of days now. Is this new behavior or has it been like this for a while? Just noticed that my ratings weren't updating for some of the newer released movies.

mynttt commented 4 years ago

Do you mean that the actual download failed because the dataset is not available anymore or that imdb isn't really updating the dataset? I remember checking the dataset before I implemented that feature and it appears that sometimes they haven't added new titles that already exist in their databases. There's nothing we can do about it however as I don't have the time to maintain a webscraping based solution at the moment.

ShrinkWrapper commented 4 years ago

ah no, the download works fine, just the information provided in the dataset is from Friday last week, so some of the ratings will be outdated with a couple of days, but its not a big issue.