search

Taxel / PlexTraktSync

A python script that syncs the movies, shows and ratings between trakt and Plex (without needing a PlexPass or Trakt VIP subscription)
MIT License
1.56k stars 101 forks source link

Syncing Wrong Shows #5

Closed bbakermmc closed 4 years ago

bbakermmc commented 4 years ago

This is syncing wrong shows that I have never watched lol

2019-11-11 20:25:16,895 INFO:Show [Crazy Claws (1981)]: Marked as watched on plex: episode S01E01
2019-11-11 20:25:17,168 INFO:Show [Crazy Claws (1981)]: Marked as watched on plex: episode S01E02
2019-11-11 20:25:17,444 INFO:Show [Crazy Claws (1981)]: Marked as watched on plex: episode S01E03
2019-11-11 20:25:17,723 INFO:Show [Crazy Claws (1981)]: Marked as watched on plex: episode S01E04
2019-11-11 20:25:18,000 INFO:Show [Crazy Claws (1981)]: Marked as watched on plex: episode S01E05
2019-11-11 20:25:18,275 INFO:Show [Crazy Claws (1981)]: Marked as watched on plex: episode S01E06
2019-11-11 20:25:18,545 INFO:Show [Crazy Claws (1981)]: Marked as watched on plex: episode S01E07
2019-11-11 20:25:18,841 INFO:Show [Crazy Claws (1981)]: Marked as watched on plex: episode S01E08
2019-11-11 20:25:19,127 INFO:Show [Crazy Claws (1981)]: Marked as watched on plex: episode S01E09
2019-11-11 20:25:19,407 INFO:Show [Crazy Claws (1981)]: Marked as watched on plex: episode S01E10
2019-11-11 20:25:19,689 INFO:Show [Crazy Claws (1981)]: Marked as watched on plex: episode S01E11
2019-11-11 20:25:19,952 INFO:Show [Crazy Claws (1981)]: Marked as watched on plex: episode S01E12
2019-11-11 20:25:20,238 INFO:Show [Crazy Claws (1981)]: Marked as watched on plex: episode S01E13
2019-11-11 20:25:20,514 INFO:Show [Crazy Claws (1981)]: Marked as watched on plex: episode S01E14
2019-11-11 20:25:20,794 INFO:Show [Crazy Claws (1981)]: Marked as watched on plex: episode S01E15
2019-11-11 20:25:21,070 INFO:Show [Crazy Claws (1981)]: Marked as watched on plex: episode S01E16
2019-11-11 20:25:21,070 INFO:Show [Crazy Claws (1981)]: Finished sync

image

bbakermmc commented 4 years ago

Maybe add a "debug" option that just spits out the log but doesnt actually do anything.

bbakermmc commented 4 years ago

Looks like its defaulting to completed = true when not found?

image

Shouldnt you be pulling the watched status from here: https://pytrakt.readthedocs.io/en/latest/users.html#trakt.users.User.watched_shows

Heres the Json of the one object looks wrong {'aired': 0, 'completed': 0, 'last_watched_at': None, 'reset_at': None, 'seasons': [], 'hidden_seasons': [], 'next_episode': None, 'last_episode': None} vs {'aired': 85, 'completed': 0, 'last_watched_at': None, 'reset_at': None, 'seasons': [{'number': 1, 'aired': 20, 'completed': 0, 'episodes': [{'number': 1, 'completed': False, 'last_watched_at': None}, {'number': 2, 'completed': False, 'last_watched_at': None}, {'number': 3, 'completed': False, 'last_watched_at': None}, {'number': 4, 'completed': False, 'last_watched_at': None}, {'number': 5, 'completed': False, 'last_watched_at': None}, {'number': 6, 'completed': False, 'last_watched_at': None}, {'number': 7, 'completed': False, 'last_watched_at': None}, {'number': 8, 'completed': False, 'last_watched_at': None}, {'number': 9, 'completed': False, 'last_watched_at': None}, {'number': 10, 'completed': False, 'last_watched_at': None}, {'number': 11, 'completed': False, 'last_watched_at': None}, {'number': 12, 'completed': False, 'last_watched_at': None}, {'number': 13, 'completed': False, 'last_watched_at': None}, {'number': 14, 'completed': False, 'last_watched_at': None}, {'number': 15, 'completed': False, 'last_watched_at': None}, {'number': 16, 'completed': False, 'last_watched_at': None}, {'number': 17, 'completed': False, 'last_watched_at': None}, {'number': 18, 'completed': False, 'last_watched_at': None}, {'number': 19, 'completed': False, 'last_watched_at': None}, {'number': 20, 'completed': False, 'last_watched_at': None}]}, {'number': 2, 'aired': 20, 'completed': 0, 'episodes': [{'number': 1, 'completed': False, 'last_watched_at': None}, {'number': 2, 'completed': False, 'last_watched_at': None}, {'number': 3, 'completed': False, 'last_watched_at': None}, {'number': 4, 'completed': False, 'last_watched_at': None}, {'number': 5, 'completed': False, 'last_watched_at': None}, {'number': 6, 'completed': False, 'last_watched_at': None}, {'number': 7, 'completed': False, 'last_watched_at': None}, {'number': 8, 'completed': False, 'last_watched_at': None}, {'number': 9, 'completed': False, 'last_watched_at': None}, {'number': 10, 'completed': False, 'last_watched_at': None}, {'number': 11, 'completed': False, 'last_watched_at': None}, {'number': 12, 'completed': False, 'last_watched_at': None}, {'number': 13, 'completed': False, 'last_watched_at': None}, {'number': 14, 'completed': False, 'last_watched_at': None}, {'number': 15, 'completed': False, 'last_watched_at': None}, {'number': 16, 'completed': False, 'last_watched_at': None}, {'number': 17, 'completed': False, 'last_watched_at': None}, {'number': 18, 'completed': False, 'last_watched_at': None}, {'number': 19, 'completed': False, 'last_watched_at': None}, {'number': 20, 'completed': False, 'last_watched_at': None}]}, {'number': 3, 'aired': 20, 'completed': 0, 'episodes': [{'number': 1, 'completed': False, 'last_watched_at': None}, {'number': 2, 'completed': False, 'last_watched_at': None}, {'number': 3, 'completed': False, 'last_watched_at': None}, {'number': 4, 'completed': False, 'last_watched_at': None}, {'number': 5, 'completed': False, 'last_watched_at': None}, {'number': 6, 'completed': False, 'last_watched_at': None}, {'number': 7, 'completed': False, 'last_watched_at': None}, {'number': 8, 'completed': False, 'last_watched_at': None}, {'number': 9, 'completed': False, 'last_watched_at': None}, {'number': 10, 'completed': False, 'last_watched_at': None}, {'number': 11, 'completed': False, 'last_watched_at': None}, {'number': 12, 'completed': False, 'last_watched_at': None}, {'number': 13, 'completed': False, 'last_watched_at': None}, {'number': 14, 'completed': False, 'last_watched_at': None}, {'number': 15, 'completed': False, 'last_watched_at': None}, {'number': 16, 'completed': False, 'last_watched_at': None}, {'number': 17, 'completed': False, 'last_watched_at': None}, {'number': 18, 'completed': False, 'last_watched_at': None}, {'number': 19, 'completed': False, 'last_watched_at': None}, {'number': 20, 'completed': False, 'last_watched_at': None}]}, {'number': 4, 'aired': 20, 'completed': 0, 'episodes': [{'number': 1, 'completed': False, 'last_watched_at': None}, {'number': 2, 'completed': False, 'last_watched_at': None}, {'number': 3, 'completed': False, 'last_watched_at': None}, {'number': 4, 'completed': False, 'last_watched_at': None}, {'number': 5, 'completed': False, 'last_watched_at': None}, {'number': 6, 'completed': False, 'last_watched_at': None}, {'number': 7, 'completed': False, 'last_watched_at': None}, {'number': 8, 'completed': False, 'last_watched_at': None}, {'number': 9, 'completed': False, 'last_watched_at': None}, {'number': 10, 'completed': False, 'last_watched_at': None}, {'number': 11, 'completed': False, 'last_watched_at': None}, {'number': 12, 'completed': False, 'last_watched_at': None}, {'number': 13, 'completed': False, 'last_watched_at': None}, {'number': 14, 'completed': False, 'last_watched_at': None}, {'number': 15, 'completed': False, 'last_watched_at': None}, {'number': 16, 'completed': False, 'last_watched_at': None}, {'number': 17, 'completed': False, 'last_watched_at': None}, {'number': 18, 'completed': False, 'last_watched_at': None}, {'number': 19, 'completed': False, 'last_watched_at': None}, {'number': 20, 'completed': False, 'last_watched_at': None}]}, {'number': 5, 'aired': 5, 'completed': 0, 'episodes': [{'number': 1, 'completed': False, 'last_watched_at': None}, {'number': 2, 'completed': False, 'last_watched_at': None}, {'number': 3, 'completed': False, 'last_watched_at': None}, {'number': 4, 'completed': False, 'last_watched_at': None}, {'number': 5, 'completed': False, 'last_watched_at': None}]}], 'hidden_seasons': [], 'next_episode': {'season': 1, 'number': 1, 'title': 'Blaze of Glory (1)', 'ids': {'trakt': 1327112, 'tvdb': 5017911, 'imdb': 'tt4461828', 'tmdb': 1131359, 'tvrage': 0}}, 'last_episode': None}

Taxel commented 4 years ago

Thanks for submitting this issue and finding a probable reason for it. I added a line that should not mark empty series as completed in commit 35af114. To reset your entire Trakt.tv collection you can use python3 clear_trakt_collections.py

Please confirm if this fixed your issue.

bbakermmc commented 4 years ago

Define "reset" entire collection. I dont want to remove my watched history in trakt, and I dont have the sync configured to do "collections" so if the reset will just purge my collected items then thats fine.

Taxel commented 4 years ago

It just removes all collected items from your Trakt library. Since you don't use that part of the sync, however, this does not help you.

bbakermmc commented 4 years ago

So my issue is the processing time, is there any way to speed it up. Why I suggested in the other one to split push/pull. And then on the push/pull only get records that are "watched" to process.

image

bbakermmc commented 4 years ago

Logs still show weirdness but it doesnt seem like its updating either end...

2019-11-12 19:41:38,781 DEBUG:Show [Chowder (2007)]: Synced episode S03E07
2019-11-12 19:41:38,781 DEBUG:Show [Chowder (2007)]: Synced episode S03E09
2019-11-12 19:41:38,782 DEBUG:Show [Chowder (2007)]: Synced episode S03E11
2019-11-12 19:41:38,782 DEBUG:Show [Chowder (2007)]: Synced episode S03E13
2019-11-12 19:41:38,782 DEBUG:Show [Chowder (2007)]: Synced episode S03E15
2019-11-12 19:41:38,782 DEBUG:Show [Chowder (2007)]: Synced episode S03E17
2019-11-12 19:41:38,782 INFO:Show [Chowder (2007)]: Finished sync
bbakermmc commented 4 years ago

Maybe I lied. Looks like its syncing it to trakt just taking time to get there .....

Taxel commented 4 years ago

Unfortunately, the processing time mainly depends on the response times of your PleX server, so without reworking the whole script (or API for that matter, since it would need to use asyncio in order to be used asynchronously) I don't see where the script performance could be improved significantly. It should work considerably faster once it has run once, though (thanks to caching the Trakt responses)

bbakermmc commented 4 years ago

Im not sure what the logs are saying when it says "Synced episode xxx"

bbakermmc commented 4 years ago

Looks like this gets called even though noting is being done.

logging.debug("Show [{} ({})]: Synced episode S{:02}E{:02}".format(
                        show.title, show.year, episode.seasonNumber, episode.index))
Taxel commented 4 years ago

Synced episode just means it has worked through the full iteration of this episode without errors, and synced the collection and watched status if specified in config. It's not really useful, that's why it's a debug and not an info log ;)

bbakermmc commented 4 years ago

New error for ya:

--- Logging error ---
Traceback (most recent call last):
  File "C:\Program Files (x86)\Python38-32\lib\logging\__init__.py", line 1084, in emit
    stream.write(msg + self.terminator)
  File "C:\Program Files (x86)\Python38-32\lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '\u0131' in position 40: character maps to <undefined>
Call stack:
  File "C:\Users\bradl\AppData\Local\JetBrains\Toolbox\apps\PyCharm-P\ch-0\193.5096.24\plugins\python\helpers\pydev\pydevd.py", line 2108, in <module>
    main()
  File "C:\Users\bradl\AppData\Local\JetBrains\Toolbox\apps\PyCharm-P\ch-0\193.5096.24\plugins\python\helpers\pydev\pydevd.py", line 2099, in main
    globals = debugger.run(setup['file'], None, None, is_module)
  File "C:\Users\bradl\AppData\Local\JetBrains\Toolbox\apps\PyCharm-P\ch-0\193.5096.24\plugins\python\helpers\pydev\pydevd.py", line 1408, in run
    return self._exec(is_module, entry_point_fn, module_name, file, globals, locals)
  File "C:\Users\bradl\AppData\Local\JetBrains\Toolbox\apps\PyCharm-P\ch-0\193.5096.24\plugins\python\helpers\pydev\pydevd.py", line 1415, in _exec
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "C:\Users\bradl\AppData\Local\JetBrains\Toolbox\apps\PyCharm-P\ch-0\193.5096.24\plugins\python\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "C:/Users/bradl/Desktop/PlexTraktSync-master/main.py", line 301, in <module>
    main()
  File "C:/Users/bradl/Desktop/PlexTraktSync-master/main.py", line 281, in main
    process_movie_section(
  File "C:/Users/bradl/Desktop/PlexTraktSync-master/main.py", line 119, in process_movie_section
    logging.info("Movie [{} ({})]: sync complete".format(
Message: 'Movie [Baskın (2015)]: sync complete'
Arguments: ()
Taxel commented 4 years ago

Completely unrelated issue, should be fixed in latest commit (I think, have never encountered this error message before but Stackoverflow is normally correct)

bbakermmc commented 4 years ago

It errors sooner now lol:

Traceback (most recent call last):
  File "C:/Users/bradl/Desktop/PlexTraktSync-master/main.py", line 301, in <module>
    main()
  File "C:/Users/bradl/Desktop/PlexTraktSync-master/main.py", line 286, in main
    process_show_section(section)
  File "C:/Users/bradl/Desktop/PlexTraktSync-master/main.py", line 165, in process_show_section
    trakt_watched = pytrakt_extensions.watched(trakt_show.slug)
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\trakt\core.py", line 472, in inner
    json_data = self._handle_request('get', url)
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\trakt\core.py", line 452, in _handle_request
    json_data = json.loads(response.content.decode('UTF-8', 'ignore'))
  File "C:\Program Files (x86)\Python38-32\lib\json\__init__.py", line 357, in loads
    return _default_decoder.decode(s)
  File "C:\Program Files (x86)\Python38-32\lib\json\decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "C:\Program Files (x86)\Python38-32\lib\json\decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Taxel commented 4 years ago

Are you sure you have pulled the latest commit? In my source files, the line where your new error occurs is line 173, not 165 like in your traceback

bbakermmc commented 4 years ago

yeah, i pulled the raw main, ran the code, and now it errors when syncing kid tv shows or something, not movies, its the section before

bbakermmc commented 4 years ago

Looks like its going further now, maybe was just a bad response or something the one time. Waiting for it to do the 15k movies lol image

bbakermmc commented 4 years ago 
Processing section Movies
Traceback (most recent call last):
  File "C:/Users/bradl/Desktop/PlexTraktSync-master/main.py", line 58, in process_movie_section
    search = trakt.sync.search_by_id(x, id_type=provider)
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\trakt\core.py", line 472, in inner
    json_data = self._handle_request('get', url)
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\trakt\core.py", line 442, in _handle_request
    response = requests.request(method, url, params=data,
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\requests\api.py", line 60, in request
    return session.request(method=method, url=url, **kwargs)
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\requests_cache\core.py", line 132, in request
    response = super(CachedSession, self).request(
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\requests\sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\requests_cache\core.py", line 109, in send
    return send_request_and_cache_response()
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\requests_cache\core.py", line 99, in send_request_and_cache_response
    self.cache.save_response(cache_key, response)
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\requests_cache\backends\base.py", line 46, in save_response
    self.responses[key] = self.reduce_response(response), datetime.utcnow()
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\requests_cache\backends\storage\dbdict.py", line 165, in __setitem__
    super(DbPickleDict, self).__setitem__(key,
  File "C:\Program Files (x86)\Python38-32\lib\site-packages\requests_cache\backends\storage\dbdict.py", line 129, in __setitem__
    con.execute("insert or replace into `%s` (key,value) values (?,?)" %
sqlite3.OperationalError: attempt to write a readonly database
Taxel commented 4 years ago

Okay, this is weird. This error stems from the trakt_cache.sqlite not being writeable, could you check the permissions on that file?

bbakermmc commented 4 years ago

I think its just because the library is so large it has issues lol. I rerun it goes for a bit, then I got the previous JSON Decode error. Its rerunning again.

bbakermmc commented 4 years ago

Yeah something is weird, it says its done syncing, but it didnt mark anything that was watched last night, run it again, completes still not updating, not sure if its because the cache is wrong or what.

2019-11-13 10:12:48,262 DEBUG:Show [After Life (2019) (None)]: Started sync
2019-11-13 10:12:48,262 DEBUG:get: https://api-v2launch.trakt.tv/search?id=360857&id_type=tvdb
2019-11-13 10:12:48,262 DEBUG:headers: {'Content-Type': 'application/json', 'trakt-api-version': '2', 'trakt-api-key': 'xxxxx', 'Authorization': 'Bearer xxxx'}
2019-11-13 10:12:48,262 DEBUG:method, url :: get, https://api-v2launch.trakt.tv/search?id=360857&id_type=tvdb
2019-11-13 10:12:48,266 DEBUG:RESPONSE [get] (https://api-v2launch.trakt.tv/search?id=360857&id_type=tvdb): <Response [200]>
2019-11-13 10:12:48,266 ERROR:Show [After Life (2019) (None)]: Did not find on Trakt. Aborting. GUID: com.plexapp.agents.thetvdb://360857?lang=en
2019-11-13 10:12:48,276 WARNING:Completed section sync in 104.05778336524963 s
2019-11-13 10:12:48,276 INFO:Updated plex watchlist
2019-11-13 10:12:48,277 INFO:Completed full sync in 603.0446376800537 seconds

Doesnt look like its going past this.

Taxel commented 4 years ago

Well it does run through every library item, so it looks fine to me, 600 seconds also sounds okay for a library that big. Your config file also looks okay to me.

bbakermmc commented 4 years ago

The last tv show in my library is not After Life (2019). I have multiple missing episodeds. EX Silicone Valley, looking in the logs, no record of it being processed, it seems like it errors and stops. I see that 13 reasons got upgdated, but appears nothing after "A". are you sure your API can pull all the values? Its like it got maybe the max for the page size.

bbakermmc commented 4 years ago

This is also why I suggested possibly splitting apart the push/pull :) I know in Plex I dont have hundreds of thousands of watched thing.

Taxel commented 4 years ago

Pagination actually might be an issue, good idea

Taxel commented 4 years ago

I have not found a reference to pagination in the plex API but added a line to the logfile for each library section where the number of movies/shows contained in the section is displayed so the result can be sanity-checked by hand. Could you pull the latest commit and check if the correct number of items is being found? If not, we have isolated the problem. If they match, it's somewhere else. I also just found a bug where the processing of a single show section stops when for a show there is no match found on Trakt, so this might fix you issue.

bbakermmc commented 4 years ago

Its processing again. Numbers look right, got another decode error, but im up to the B's in TV shows, since i see like 20 A series were marked.

bbakermmc commented 4 years ago 
Traceback (most recent call last):
  File "C:/Users/bradl/Desktop/PlexTraktSync-master/main.py", line 188, in process_show_section
    eps = lookup[episode.seasonNumber][episode.index]
KeyError: None

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:/Users/bradl/Desktop/PlexTraktSync-master/main.py", line 311, in <module>
    main()
  File "C:/Users/bradl/Desktop/PlexTraktSync-master/main.py", line 296, in main
    process_show_section(section)
  File "C:/Users/bradl/Desktop/PlexTraktSync-master/main.py", line 228, in process_show_section
    logging.warning("Show [{} ({})]: Key not found, did not record episode S{:02}E{:02}".format(
TypeError: unsupported format string passed to NoneType.__format__
Taxel commented 4 years ago

Okay perfect, this is the error in #6 Closing this issue now.