search

Taxel / PlexTraktSync

A python script that syncs the movies, shows and ratings between trakt and Plex (without needing a PlexPass or Trakt VIP subscription)
MIT License
1.61k stars 105 forks source link

Errors from 0.6.40 #220

Closed BEisem closed 3 years ago

BEisem commented 3 years ago

I downloaded https://github.com/Taxel/PlexTraktSync/pull/206 to test out, and I'm running into some errors.

If I try to sync only movie categories (by excluding all other libraries in config.json I get a long pause, and finally a crash with the following output:

INFO: Syncing with Plex BlaEis and Trakt BlaEis
INFO: Syncing TV=True, Movies=True
INFO: Loaded Trakt lists in 3.9 seconds
INFO: Downloaded List Trakt Watchlist
INFO: Downloaded List Rotten Tomatoes: Best of 2019
INFO: Downloaded List IMDB: Top Rated Movies
INFO: Server version 1.22.2.4282-a97b03fad updated at: 2021-04-10 13:51:57
INFO: Recently added: [<Movie:123435:Billy-Connolly:-Live>, <Movie:196499:Lazarus>, <Season:196476:7th-Heaven-s9>, <Season:196454:7th-Heaven-s3>, <Season:196431:7th-Heaven-s4>]
Traceback (most recent call last):
  File "main.py", line 6, in <module>
    cli()
  File "/Library/Python/3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/Library/Python/3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/Library/Python/3.7/site-packages/click/core.py", line 1236, in invoke
    return Command.invoke(self, ctx)
  File "/Library/Python/3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Library/Python/3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/Library/Python/3.7/site-packages/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/Users/bink/scripts/PlexTraktSync/plex_trakt_sync/cli.py", line 14, in cli
    sync()
  File "/Library/Python/3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/Library/Python/3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/Library/Python/3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Library/Python/3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/Users/bink/scripts/PlexTraktSync/plex_trakt_sync/commands/sync.py", line 193, in sync
    sync_all(movies=movies, tv=tv)
  File "/Users/bink/scripts/PlexTraktSync/plex_trakt_sync/commands/sync.py", line 154, in sync_all
    for pm, tm in for_each_pair(plex.movie_sections, trakt):
  File "/Users/bink/scripts/PlexTraktSync/plex_trakt_sync/commands/sync.py", line 97, in for_each_pair
    for pm in section.items():
  File "/Users/bink/scripts/PlexTraktSync/plex_trakt_sync/plex_api.py", line 116, in items
    if item.provider in ["local", "none", "agents.none"]:
  File "/Users/bink/scripts/PlexTraktSync/plex_trakt_sync/plex_api.py", line 43, in provider
    x = self.guid.split("://")[0]
  File "/Users/bink/scripts/PlexTraktSync/plex_trakt_sync/plex_api.py", line 18, in guid
    if len(self.item.guids) > 0:
  File "/Library/Python/3.7/site-packages/plexapi/base.py", line 441, in __getattribute__
    self.reload()
  File "/Library/Python/3.7/site-packages/plexapi/base.py", line 323, in reload
    data = self._server.query(key)
  File "/Library/Python/3.7/site-packages/plexapi/server.py", line 531, in query
    raise NotFound(message)
plexapi.exceptions.NotFound: (404) not_found; https://10-243-234-60.8ffa95f9a6a4493ba0a24465abed823e.plex.direct:32400/library/metadata/147929?checkFiles=1&includeAllConcerts=1&includeBandwidths=1&includeChapters=1&includeChildren=1&includeConcerts=1&includeExternalMedia=1&includeExtras=1&includeFields=thumbBlurHash%2CartBlurHash&includeGeolocation=1&includeLoudnessRamps=1&includeMarkers=1&includeOnDeck=1&includePopularLeaves=1&includePreferences=1&includeRelated=1&includeRelatedCount=1&includeReviews=1&includeStations=1 <html><head><title>Not Found</title></head><body><h1>404 Not Found</h1></body></html>

If I do TV libraries, I get another error, which I will post in a comment just to keep them separated.

BEisem commented 3 years ago

With TV libraries only, I get this error much quicker (without much delay at all).

INFO: Syncing with Plex BlaEis and Trakt BlaEis
INFO: Syncing TV=True, Movies=True
INFO: Loaded Trakt lists in 3.8 seconds
INFO: Downloaded List Trakt Watchlist
INFO: Downloaded List Rotten Tomatoes: Best of 2019
INFO: Downloaded List IMDB: Top Rated Movies
INFO: Server version 1.22.2.4282-a97b03fad updated at: 2021-04-10 15:14:19
INFO: Recently added: [<Movie:123435:Billy-Connolly:-Live>, <Movie:196501:Breaking-News-in-Yub>, <Movie:196499:Lazarus>, <Season:196476:7th-Heaven-s9>, <Season:196454:7th-Heaven-s3>]
Traceback (most recent call last):
  File "/Users/bink/scripts/PlexTraktSyncTV/plex_trakt_sync/trakt_api.py", line 190, in find_movie
    search = trakt.sync.search_by_id(media.id, id_type=media.provider, media_type=media.media_type)
  File "/Library/Python/3.7/site-packages/trakt/core.py", line 542, in inner
    resp = self._get_first(f, *args, **kwargs)
  File "/Library/Python/3.7/site-packages/trakt/core.py", line 489, in _get_first
    uri = next(generator)
  File "/Library/Python/3.7/site-packages/trakt/sync.py", line 255, in search_by_id
    raise ValueError('search_type must be one of {}'.format(valids))
ValueError: search_type must be one of ('trakt', 'trakt-movie', 'trakt-show', 'trakt-episode', 'trakt-person', 'imdb', 'tmdb', 'tvdb')

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "main.py", line 6, in <module>
    cli()
  File "/Library/Python/3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/Library/Python/3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/Library/Python/3.7/site-packages/click/core.py", line 1236, in invoke
    return Command.invoke(self, ctx)
  File "/Library/Python/3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Library/Python/3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/Library/Python/3.7/site-packages/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/Users/bink/scripts/PlexTraktSyncTV/plex_trakt_sync/cli.py", line 14, in cli
    sync()
  File "/Library/Python/3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/Library/Python/3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/Library/Python/3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Library/Python/3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/Users/bink/scripts/PlexTraktSyncTV/plex_trakt_sync/commands/sync.py", line 193, in sync
    sync_all(movies=movies, tv=tv)
  File "/Users/bink/scripts/PlexTraktSyncTV/plex_trakt_sync/commands/sync.py", line 160, in sync_all
    for pm, tm, pe, te in for_each_episode(plex.show_sections, trakt):
  File "/Users/bink/scripts/PlexTraktSyncTV/plex_trakt_sync/commands/sync.py", line 111, in for_each_episode
    for pm, tm in for_each_pair(sections, trakt):
  File "/Users/bink/scripts/PlexTraktSyncTV/plex_trakt_sync/commands/sync.py", line 102, in for_each_pair
    tm = trakt.find_movie(pm)
  File "/Users/bink/scripts/PlexTraktSyncTV/plex_trakt_sync/decorators/rate_limit.py", line 43, in wrapper
    return fn(*args, **kwargs)
  File "/Users/bink/scripts/PlexTraktSyncTV/plex_trakt_sync/trakt_api.py", line 195, in find_movie
    raise ValueError(f"Invalid id_type: {media.provider}") from e
ValueError: Invalid id_type: plex
BEisem commented 3 years ago

Also, I should point out that the script is reporting INFO: Syncing with Plex BlaEis and Trakt BlaEis while my Trakt username is actually something else. In the master branch, it reports the correct Trakt username. It seems like this version is using my Plex username for both. I don't know if that is the cause of this error, or if it is just a display error. But I wanted to point it out.

Also, I should point out that I am using the new Plex agents for both TV and Movie libraries.

glensc commented 3 years ago

@BEisem I don't understand what did you checkout (no details) and what did you executed (no details). you refer to #206, but that is merged and branch deleted...

please, include the used git tag and the actual command being executed.

BEisem commented 3 years ago

I did what you said ... I changed to the main branch then downloaded the zip file. Installed to my server, then ran python3 main.py and the result was what I posted above. (All the items in requirements.txt are already installed, so I didn't have to do any updates).

BEisem commented 3 years ago

You've mentioned "checkout" but I don't understand how that works. Also, if it's possible to do a git clone command directly using a specific tag, I don't know how to do that. What would be the command to do that directly?

glensc commented 3 years ago

@BEisem If you mean by my instructions this:

Then these instructions do not apply for #206, as it's merged, so the source link does not point to the actual branch anymore, but just to my fork. so it's anything else, but not what #206 was merging.

How to git checkout specific tag, was answered here:

do read the proposed changes. use view file from the menu if you're unable to read the diffs

image

and to show what you were executing, post full command as well what you were executing.

so now you are saying you used main branch, then #206 has nothing to do with what you were testing? if you be testing #206 only, I'd expect you to use --sync=tv or --sync=movies option.

glensc commented 3 years ago
  1. the error with movie:

looks like something was deleted meanwhile the script was running. not sure what to do here, other than log and ignore such errors.

  1. the error with tv

need more details, what is the guid of the failing item? apply this change and show error line again.

diff --git a/plex_trakt_sync/trakt_api.py b/plex_trakt_sync/trakt_api.py
index 06361a7..2a0dbd7 100644
--- a/plex_trakt_sync/trakt_api.py
+++ b/plex_trakt_sync/trakt_api.py
@@ -205,7 +205,7 @@ def find_movie(self, media: PlexLibraryItem):
             raise ValueError(f"Unable parse search result for {media.provider}/{media.id}: {e.doc!r}") from e
         except ValueError as e:
             # Search_type must be one of ('trakt', ..., 'imdb', 'tmdb', 'tvdb')
-            raise ValueError(f"Invalid id_type: {media.provider}") from e
+            raise ValueError(f"Invalid id_type: {media.provider} guid: {media.guid}") from e
         # look for the first wanted type in the results
         for m in search:
             if m.media_type == media.type:
BEisem commented 3 years ago

Ok I made the change, here is the new error (only the bottom part that changed):

ValueError: Invalid id_type: plex guid: plex://show/5d9c08053c3f87001f345254

BEisem commented 3 years ago

Incidentally if I do python3 main.py --sync=tv I get an error as well:

Usage: main.py [OPTIONS] COMMAND [ARGS]...
Try 'main.py --help' for help.

Error: no such option: --sync
glensc commented 3 years ago

Okay, this should do the trick:

apply the diff from there, or checkout that branch

glensc commented 3 years ago

Incidentally if I do python3 main.py --sync=tv I get an error as well:

Usage: main.py [OPTIONS] COMMAND [ARGS]...
Try 'main.py --help' for help.

Error: no such option: --sync```
  1. again you misplaced markdown close backticks, there needs to be a new line in front
  2. readme was updated in #206, includes correct usage

EDIT: readme direct link https://github.com/glensc/plex-trakt-sync/blob/6126319154b84af265a566f5f9500b770035a370/README.md#sync-options

BEisem commented 3 years ago

Okay, this should do the trick:

  • 222

apply the diff from there, or checkout that branch

Ok, I made that change. Now I'm getting the following error:

ValueError: Invalid id_type: plex, guid: plex://show/5d9c085ae98e47001eb0d74f

By the way the command I am issuing is python3 main.py. I also tried python3 main.py sync --sync=tv and got the same result.

glensc commented 3 years ago

Okay, this should do the trick:

  • 222

apply the diff from there, or checkout that branch

Ok, I made that change. Now I'm getting the following error:

ValueError: Invalid id_type: plex, guid: plex://show/5d9c085ae98e47001eb0d74f

I don't think you did it right. or did not apply all changes.

The branch link (if you click the source branch) leads to;

did you download the zip from there? or used .diff/.patch? or used git?

By the way the command I am issuing is python3 main.py. I also tried python3 main.py sync --sync=tv and got the same result.

yes, main.py without args just syncs both.

BEisem commented 3 years ago

Ok, thank you... I downloaded the zip from https://github.com/glensc/plex-trakt-sync/tree/tv-agent-sync-fix and tried again, but unfortunately I'm still getting an error. It runs through a few shows, but then crashes with the error

ValueError: Invalid id_type: 'plex', guid: 'plex://show/5d9c085ae98e47001eb0d74f'.

It seems to be that same show every time. I'm not sure how to determine what show it is, though.

BEisem commented 3 years ago
  1. the error with movie:

looks like something was deleted meanwhile the script was running. not sure what to do here, other than log and ignore such errors.

The error with the movie continues to happen every time I run python3 main.py sync --sync=movies ... nothing is being deleted during the scan.

glensc commented 3 years ago

ValueError: Invalid id_type: 'plex', guid: 'plex://show/5d9c085ae98e47001eb0d74f'.

It seems to be that same show every time. I'm not sure how to determine what show it is, though.

@BEisem change the exception to:

raise ValueError(f"Invalid id_type: '{media.provider}', guid: '{media.guid}', guids:'{media.item.guids}: {e}") from e

EDIT: added {e} to message

wandabastyle commented 3 years ago

So I did also testing with this (incl. the change in the post above) and all in all it seems to work. All warnings I am catching are related to incosistend data between trakt and the metadata providers (imdb/tmdb/tvdb) or my shitty sorting of some specials.

There is still an ValueError for files without metadata.

ValueError: Invalid id_type: 'tv.plex.agents.none', guid: 'tv.plex.agents.none://68178', guids:'[]

glensc commented 3 years ago

tv.plex.agents.none looks fishy, how many variants there are for "none"? Anyway, added fix for that into https://github.com/Taxel/PlexTraktSync/pull/222

glensc commented 3 years ago

The error with the movie continues to happen every time I run python3 main.py sync --sync=movies ... nothing is being deleted during the scan.

see what's up with your media item: /library/metadata/147929

probably at https://app.plex.tv/desktop/#!/server/8ffa95f9a6a4493ba0a24465abed823e/details?key=/library/metadata/147929

wandabastyle commented 3 years ago

I got a full sync. Now I have to sort out the warning. Will check the full log later. It's kinda late today. Thx for your work. Are all these fixes already part of #209?

BEisem commented 3 years ago

I've been running the new version on my movie library, went out to grab some dinner and it's been running for about an hour and a half with no output. I noticed that this version does not include any "pass" output, only errors. Is this intentional? There is no way to tell that it's still working other than to check the CPU usage of python, so I was wondering if it would make sense to add some sort of indication (in the logs or in the output) that it is still working.

BEisem commented 3 years ago

Small grammar error ... in the logs, WARNING: [<imdb:tt8892066:<Movie:147427:Big-Blue-Sea>>)]: Not found from Trakt. Skipping should say Not found on Trakt

BEisem commented 3 years ago

@BEisem change the exception to:

raise ValueError(f"Invalid id_type: '{media.provider}', guid: '{media.guid}', guids:'{media.item.guids}: {e}") from e

EDIT: added {e} to message

I've made this change, but I'm still unable to run a TV scan. With the change listed above, the error is now:

ValueError: Invalid id_type: 'plex', guid: 'plex://show/5d9c085ae98e47001eb0d74f', guids: '[]': search_type must be one of ('trakt', 'trakt-movie', 'trakt-show', 'trakt-episode', 'trakt-person', 'imdb', 'tmdb', 'tvdb')

glensc commented 3 years ago

@BEisem good output, there's nothing we can to do with plex://show/5d9c085ae98e47001eb0d74f, because it has no usable external id at all. i.e the .guid is plex internal and the newer .guids is empty.

I'll add skip to such invalid entries.

As for the progress bar, it's added on 0.7.x branch: https://github.com/Taxel/PlexTraktSync/pull/212

@wandabastyle you should be able to see commits from https://github.com/Taxel/PlexTraktSync/pull/209 yourself, which are new, i.e:

I'll add changes from https://github.com/Taxel/PlexTraktSync/pull/222 there now. as seems it fixes some issues.

glensc commented 3 years ago

@BEisem I've added skip of the movie that you are getting a not found error:

glensc commented 3 years ago

I've also added a logic change that all invalid items be skipped early:

BEisem commented 3 years ago

This is great, TV scan is now working ... I'm in the process of doing an initial scan and it is progressing without errors.

One thing I noticed is that in between every line indicating something is being added to Trakt, I am also getting a warning:

WARNING: Sleeping for 0.132 seconds

I think if it is going to sleep in between each scan, it shouldn't necessarily notify us of that, as it makes the logs unnecessarily long and messy. Maybe only report it if it is above a certain length? Or if it has to sleep repeatedly due to an API issue?

Thank you for the work! This is working great now...

simonc56 commented 3 years ago

I already warned about it a week ago :

You should remove the "Sleeping for x sec" logging because it is a normal behaviour and it will create too much noise in logs. Or at least move to debug level.

https://github.com/Taxel/PlexTraktSync/pull/177#issuecomment-813960645

glensc commented 3 years ago

@BEisem is there still something remaining to be solved in this issue or you can close the issue?

glensc commented 3 years ago

I already warned about it a week ago :

You should remove the "Sleeping for x sec" logging because it is a normal behaviour and it will create too much noise in logs. Or at least move to debug level.

#177 (comment)

warn changed to debug: #229

glensc commented 3 years ago

One thing I noticed is that in between every line indicating something is being added to Trakt, I am also getting a warning:

WARNING: Sleeping for 0.132 seconds

I think if it is going to sleep in between each scan, it shouldn't necessarily notify us of that, as it makes the logs unnecessarily long and messy. Maybe only report it if it is above a certain length? Or if it has to sleep repeatedly due to an API issue?

Implemented batching add_to_collection trakt api calls: