mynttt / UpdateTool

A tool to update the IMDB ratings for Plex libraries that contain movies/series and use the IMDB agent to receive ratings
GNU General Public License v3.0
257 stars 12 forks source link

IMDB item with id {id} appears to not be allowed to be rated by anyone. #68

Closed JeremyKennedy closed 3 years ago

JeremyKennedy commented 3 years ago

Hey, I am starting up this tool for the first time and am seeing some weird behavior with my TV library.

Both of my TV+Movie libraries are using the "new" Plex agents and IMDb as the rating source. I have a paid TVDB API key and a TMDB API key. Running UpdateTool with "NO_TV" mode it works as expected. But when I run it on the TV library (using the opt-in environment variable), my logs fill up with the below:

[INFO ] - 2021-09-04 14:29:18 @ ImdbScraper.scrape: IMDB item with id tt7320236 appears to not be allowed to be rated by anyone. Ignoring for 30 day(s).
[INFO ] - 2021-09-04 14:29:19 @ ImdbScraper.scrape: IMDB item with id tt4771198 appears to not be allowed to be rated by anyone. Ignoring for 30 day(s).
[INFO ] - 2021-09-04 14:29:20 @ ImdbScraper.scrape: IMDB item with id tt8114272 appears to not be allowed to be rated by anyone. Ignoring for 30 day(s).
[INFO ] - 2021-09-04 14:29:21 @ ImdbScraper.scrape: IMDB item with id tt4398274 appears to not be allowed to be rated by anyone. Ignoring for 30 day(s).
[INFO ] - 2021-09-04 14:29:22 @ ImdbScraper.scrape: IMDB item with id tt4398276 appears to not be allowed to be rated by anyone. Ignoring for 30 day(s).
[INFO ] - 2021-09-04 14:29:23 @ ImdbScraper.scrape: IMDB item with id tt5445684 appears to not be allowed to be rated by anyone. Ignoring for 30 day(s).
[INFO ] - 2021-09-04 14:29:25 @ ImdbScraper.scrape: IMDB item with id tt5603948 appears to not be allowed to be rated by anyone. Ignoring for 30 day(s).
[INFO ] - 2021-09-04 14:29:26 @ ImdbScraper.scrape: IMDB item with id tt4276084 appears to not be allowed to be rated by anyone. Ignoring for 30 day(s).
[INFO ] - 2021-09-04 14:29:27 @ ImdbScraper.scrape: IMDB item with id tt4276082 appears to not be allowed to be rated by anyone. Ignoring for 30 day(s).

Consistently, every second, it spits a line like this out, and when I search IMDb for the referenced ID, it is an item that I have nowhere in my Plex library. Is this normal? It has been doing this for about 45 minutes now.

My library is massive so I am not sure if I should leave this running or if something is wrong. I have 1370 movies, 20000 TV episodes, and my database is almost 300 MB. I think I will stop it out of fear that this is not normal.

Please let me know if you need any more information!

mynttt commented 3 years ago

Looks like it is always using the web scraper fallback for some reason. That should only be used when an ID is not in the official IMDB rating set. The IDs that appear in the log don't seem to have a rating associated on the web page either.

I would have to check your database to understand why this happens. If you want you can send it to my email address: marc@herschel.io

But it'll probably take some while until I find time to look into it!

JeremyKennedy commented 3 years ago

Just emailed the link to you. I'm familiar with databases, just not the Plex db or what UpdateTool depends on. If there's anything I can check on my own in the meantime, please let me know!

simonk83 commented 3 years ago

I just started using this yesterday and was also seeing lines like this in the log. My docker command is:

sudo docker run -dit -e UNLOCK_FOR_NEW_TV_AGENT="2;3;7" -e IGNORE_LIBS="6" -e TVDB_API_KEY="****" -v "/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/":/plexdata -v "/Docker/imdpupdaterconfig":/config mynttt/updatetool

[INFO ] - 2021-09-04 22:25:37 @ ImdbScraper.scrape: IMDB item with id tt5347840 appears to not be allowed to be rated by anyone. Ignoring for 30 day(s). [INFO ] - 2021-09-04 22:25:40 @ ImdbScraper.scrape: IMDB item with id tt5541866 appears to not be allowed to be rated by anyone. Ignoring for 30 day(s). [INFO ] - 2021-09-04 22:25:42 @ ImdbScraper.scrape: IMDB item with id tt5347830 appears to not be allowed to be rated by anyone. Ignoring for 30 day(s). [INFO ] - 2021-09-04 22:25:45 @ ImdbScraper.scrape: IMDB item with id tt5169260 appears to not be allowed to be rated by anyone. Ignoring for 30 day(s). [INFO ] - 2021-09-04 22:25:47 @ ImdbScraper.scrape: IMDB item with id tt5347836 appears to not be allowed to be rated by anyone. Ignoring for 30 day(s). [INFO ] - 2021-09-04 22:25:50 @ ImdbScraper.scrape: IMDB item with id tt5169270 appears to not be allowed to be rated by anyone. Ignoring for 30 day(s). [INFO ] - 2021-09-04 22:25:52 @ ImdbScraper.scrape: IMDB item with id tt4932898 appears to not be allowed to be rated by anyone. Ignoring for 30 day(s). [INFO ] - 2021-09-04 22:25:55 @ ImdbScraper.scrape: IMDB item with id tt5545650 appears to not be allowed to be rated by anyone. Ignoring for 30 day(s). [INFO ] - 2021-09-04 22:25:57 @ ImdbScraper.scrape: IMDB item with id tt5169280 appears to not be allowed to be rated by anyone. Ignoring for 30 day(s). [INFO ] - 2021-09-04 22:26:00 @ ImdbScraper.scrape: IMDB item with id tt5614988 appears to not be allowed to be rated by anyone. Ignoring for 30 day(s). [INFO ] - 2021-09-04 22:26:02 @ ImdbScraper.scrape: IMDB item with id tt5609788 appears to not be allowed to be rated by anyone. Ignoring for 30 day(s). [INFO ] - 2021-09-04 22:26:05 @ ImdbScraper.scrape: IMDB item with id tt7057698 appears to not be allowed to be rated by anyone. Ignoring for 30 day(s). [INFO ] - 2021-09-04 22:26:07 @ ImdbScraper.scrape: IMDB item with id tt5169266 appears to not be allowed to be rated by anyone. Ignoring for 30 day(s). [INFO ] - 2021-09-04 22:26:09 @ ImdbScraper.scrape: IMDB item with id tt5574890 appears to not be allowed to be rated by anyone. Ignoring for 30 day(s). [INFO ] - 2021-09-04 22:26:12 @ ImdbScraper.scrape: IMDB item with id tt5717680 appears to not be allowed to be rated by anyone. Ignoring for 30 day(s). [INFO ] - 2021-09-04 22:26:12 @ ImdbPipeline.transformMetadata: 15 item(s) need no update.

These (or at least the ones I checked) all seem to be part of my kids library I just added. I've no idea what's different about those files, they don't seem to be locked in Plex or anything.

EDIT: Oh I sorry I missed your comment above mentioned that these items don't seem to have a rating on IMDB anyway. Is it just because of that?

I also see a fair few of these:

DefaultResolvement.resolve: Item: '' has no matching IMDB resolver and will be ignored. (guid=local://3674) [WARN ] - 2021-09-04 22:25:34 @ DefaultResolvement.resolve: Item: '' has no matching IMDB resolver and will be ignored. (guid=local://4504) [WARN ] - 2021-09-04 22:25:34 @ DefaultResolvement.resolve: Item: '' has no matching IMDB resolver and will be ignored. (guid=local://4505) [WARN ] - 2021-09-04 22:25:34 @ DefaultResolvement.resolve: Item: '' has no matching IMDB resolver and will be ignored. (guid=local://4506) [WARN ] - 2021-09-04 22:25:34 @ DefaultResolvement.resolve: Item: '' has no matching IMDB resolver and will be ignored. (guid=local://4507) [WARN ] - 2021-09-04 22:25:34 @ DefaultResolvement.resolve: Item: '' has no matching IMDB resolver and will be ignored. (guid=local://4509) [WARN ] - 2021-09-04 22:25:34 @ DefaultResolvement.resolve: Item: '' has no matching IMDB resolver and will be ignored. (guid=local://4510) Presumably that's an item without a title or something like that (seems odd though if that's the case) but I'm not sure how to track down what item those actually are.

EDIT: Ok I found them by going into XML view and dropping in that GUID number. Looks like some of these are just weird episodes that sort of exist, but not really? Eg Episode 6 of this: https://www.imdb.com/title/tt0168596/episodes?season=5, but then there's one's like this (local://26220) that seem fine on imdb, so I'm not sure what the problem is afterall: https://www.imdb.com/title/tt6885982/?ref_=ttep_ep1

JeremyKennedy commented 3 years ago

So I haven't looked further into UpdateTool or this issue, but it does seem like my database got corrupted when I was trying to get this working earlier. The corruption symptoms were that some (not all) TV shows failed to play with error "An error occurred loading items to play". I was luckily able to repair it with no data loss (of course, I have daily backups too). So, @mynttt I believe the database I've sent you is corrupted.

mynttt commented 3 years ago

@simonk83 The warning has nothing to do with Plex itself, the string is a bit misleading. The tool is unable to resolve the matched IMDB ID with the public dataset, as the dataset does not contain a rating for this item. That happens mostly with unrated items. There is a screen scraping module for backup that tries to scrape the rating from the IMDB website as in some rare cases an item is not included in the dataset that is refreshed daily by IMDB. Some IMDB items are not open for rating tho (mostly strange and obscure episodes afaik). So the tool spits out this message to inform that it is likely that there won't be any rating for them anytime soon. Might be a good idea to introduce a flag to stop this log message from appear.

The item with '' is most likely some strange episode, as you said. The problem is that the guid local://4510 makes it impossible for this tool to identify the corresponding IMDB ID. So it spits out this warning and does nothing.

@JeremyKennedy Interesting to know! I'll check the DB file before investigating it, if it's corrupt I can also try to repair it first via SQLites repair commands. If that fails, I'd let you know! However I won't have time to look at this until the 20th September as I still have an exam to complete until then.

JeremyKennedy commented 3 years ago

@mynttt I didn't have success with the "recommended" repair command: "Plex SQLite" com.plexapp.plugins.library.db ".output recover.out" ".recover" but I did get it back with the "Manually Repair the Database" steps here which dumps to sql and rebuilds. Not sure why, but hey, I'm glad it is back to normal now. No worries about the delay, good luck with your exam!

mynttt commented 3 years ago

@JeremyKennedy Regarding the SQLite corruption issue with 1.6.0 there is a new patch that applies write operations to the database via the non-standard SQLite binary that is compiled and supplied by Plex. To enable this feature set the docker variable USE_PLEX_SQLITE_BINARY_FOR_WRITE_ACCESS to true and update the docker.

mynttt commented 3 years ago

@JeremyKennedy You can use IGNORE_SCRAPER_NO_RESULT_LOG in combination with the CAPABILITIES variable to supress those web scraper messages. They're harmless (the targeted items just have no ratings on the IMDB website).