search

Taxel / PlexTraktSync

A python script that syncs the movies, shows and ratings between trakt and Plex (without needing a PlexPass or Trakt VIP subscription)
MIT License
1.61k stars 105 forks source link

Feature request: Add HAMA provider #532

Closed eduardlleshi closed 3 years ago

eduardlleshi commented 3 years ago

Feature description

It seems like any anime Movie or TV Show is not able to be synced to Trakt due to HAMA being its provider.

I'm not sure if that's a limitation on Plex or Trakt, but it would be awesome this is something that can be added on PlexTraktSync in the near future.

HAMA stands for HTTP AniDB Metadata Agent: https://github.com/ZeroQI/Hama.bundle AniDB is an Anime Database that contains more anime than TVDB: https://anidb.net/

These are two sample errors (one fetching from AniDB and another from TVDB). I can provide more debugging if necessary.

<Show:785:One-Punch-Man>: Unable to parse a valid provider from guid com.plexapp.agents.hama://anidb-11123?lang=en
<Show:839:My-Hero-Academia>: Unable to parse a valid provider from guid com.plexapp.agents.hama://tvdb-305074?lang=en

Use case

Running sync or watch would match Anime media and sync the status to Trakt.

Plan to implement

I'd love to contribute, but unfortunately, I don't have experience with Python.

glensc commented 3 years ago

Add edit description to links and description what is Hama, anidb

eduardlleshi commented 3 years ago

I've updated the description to include links to HAMA and AniDB.

glensc commented 3 years ago

@eduardlleshi Are you willing to add a section to README.md on how to install and use this plugin?

glensc commented 3 years ago

@eduardlleshi can you please also paste example filenames for movies and series in form that plex understands. so could test the agent with empty files locally.

eduardlleshi commented 3 years ago

@glensc I've followed the instructions on the HAMA repo to get both Absolute Series Scanner (as a scanner) and HAMA (as an agent) installed. I'm running the Plex Server on an Nvidia Shield TV which had its complexities to get these installed.

My Library setup looks like this: Image 2021-10-14 at 11 00 08

I've added the filenames to the series here.

What's interesting is that with a previous version (0.15.0) the errors contained more info on the GUIDs, including IMBD and TVDB. Would that help in matching the animes without the need for AniDB API?

ERROR: <hama:anidb-11123:<Show:785:One-Punch-Man>>: Unable to parse a valid provider from guid:'com.plexapp.agents.hama://anidb-11123?lang=en', guids:[<Guid:imdb://tt4508902>, <Guid:tmdb://63926>, <Guid:tvdb://293088>]
ERROR: <hama:tvdb-305074:<Show:839:My-Hero-Academia>>: Unable to parse a valid provider from guid:'com.plexapp.agents.hama://tvdb-305074?lang=en', guids:[<Guid:imdb://tt5626028>, <Guid:tmdb://65930>, <Guid:tvdb://305074>]
glensc commented 3 years ago

What's interesting is that with a previous version (0.15.0) the errors contained more info on the GUIDs, including IMBD and TVDB. Would that help in matching the animes without the need for AniDB API?

ERROR: <hama:anidb-11123:<Show:785:One-Punch-Man>>: Unable to parse a valid provider from guid:'com.plexapp.agents.hama://anidb-11123?lang=en', guids:[<Guid:imdb://tt4508902>, <Guid:tmdb://63926>, <Guid:tvdb://293088>]
ERROR: <hama:tvdb-305074:<Show:839:My-Hero-Academia>>: Unable to parse a valid provider from guid:'com.plexapp.agents.hama://tvdb-305074?lang=en', guids:[<Guid:imdb://tt5626028>, <Guid:tmdb://65930>, <Guid:tvdb://305074>]

do share what is the current version and current outcome?

also, looks like your initial issue description lacks version information, please fill that in as well.

also, when i asked about writing documentation:

I meant adding it to readme.md via pull request.

simonc56 commented 3 years ago

You talk about a sync error :

It seems like any anime Movie or TV Show is not able to be synced to Trakt due to HAMA being its provider.

Here we have logs about a provider error :

<Show:785:One-Punch-Man>: Unable to parse a valid provider from guid com.plexapp.agents.hama://anidb-11123?lang=en
<Show:839:My-Hero-Academia>: Unable to parse a valid provider from guid com.plexapp.agents.hama://tvdb-305074?lang=en

But episodes have many guids, so this error about one guid should not prevent the sync. Other guids (imdb, tvdb) should match and make the sync work.

glensc commented 3 years ago

@eduardlleshi please add examples of HAMA movies.

Run inspect command, for example:

./plex_trakt_sync.sh inspect 839

the number is plex key from url:

glensc commented 3 years ago

Added some hack based on local tests with no media:

but could do better with inspect output of speciic items, as I get just guid=xxx with empty guids with agent setup I did locally:

Frazzer951 commented 3 years ago

@eduardlleshi please add examples of HAMA movies.

Run inspect command, for example:

./plex_trakt_sync.sh inspect 839

the number is plex key from url:

  • details?key=%2Flibrary%2Fmetadata%2F839

Ran plex_trakt_sync inspect on a couple of different things, those being a Movie, a Show, and an Episode, that all have HAMA as their agent and here are the results.

Movie 'The Cat Returns' with details?key=%2Flibrary%2Fmetadata%2F37017

PlexTraktSync inspect [0.15.2]
Inspecting 37017: <hama:anidb-679:<Movie:37017:The-Cat-Returns>>
URL: https://app.plex.tv/desktop/#!/server/8c370d3afeaf4edfd23b0ed8e023008b43c8414d/details?key=/library/metadata/37017
Media.Type: movie
Media.Guid: 'com.plexapp.agents.hama://anidb-679?lang=en'
Audio: 'stereo', 'English (AAC Stereo)'
Video: 'hevc'
Guids:
  Guid: com.plexapp.agents.hama://anidb-679?lang=en, Id: anidb-679, Provider: hama
Metadata: {'collected_at': '2021-04-20:T18:04:59.000Z', 'media_type': 'digital', 'resolution': 'hd_1080p', 'audio': 'dts', 'audio_channels': '5.1'}
ERROR: <Movie:37017:The-Cat-Returns>: Unable to parse a valid provider from guid com.plexapp.agents.hama://anidb-679?lang=en

Show 'Samurai Champloo' with details?key=%2Flibrary%2Fmetadata%2F41903

PlexTraktSync inspect [0.15.2]
Inspecting 41903: <hama:anidb-1543:<Show:41903:Samurai-Champloo>>
URL: https://app.plex.tv/desktop/#!/server/8c370d3afeaf4edfd23b0ed8e023008b43c8414d/details?key=/library/metadata/41903
Media.Type: show
Media.Guid: 'com.plexapp.agents.hama://anidb-1543?lang=en'
Guids:
  Guid: com.plexapp.agents.hama://anidb-1543?lang=en, Id: anidb-1543, Provider: hama
Metadata: {'collected_at': '2020-09-07:T02:26:07.000Z', 'media_type': 'digital'}
ERROR: <Show:41903:Samurai-Champloo>: Unable to parse a valid provider from guid com.plexapp.agents.hama://anidb-1543?lang=en

Episode 'Samuai Champloo Ep 1' with details?key=%2Flibrary%2Fmetadata%2F41905

PlexTraktSync inspect [0.15.2]
Inspecting 41905: <hama:anidb-1543/1/1:<Episode:41905:Samurai-Champloo-s01e01>>
URL: https://app.plex.tv/desktop/#!/server/8c370d3afeaf4edfd23b0ed8e023008b43c8414d/details?key=/library/metadata/41905
Media.Type: episode
Media.Guid: 'com.plexapp.agents.hama://anidb-1543/1/1?lang=en'
Audio: 'stereo', 'English (AAC Stereo)'
Video: 'hevc'
Guids:
  Guid: com.plexapp.agents.hama://anidb-1543/1/1?lang=en, Id: anidb-1543/1/1, Provider: hama
Metadata: {'collected_at': '2021-04-20:T18:07:47.000Z', 'media_type': 'digital', 'resolution': 'hd_720p', 'audio': 'aac', 'audio_channels': '2.0'}
ERROR: <Episode:41905:Samurai-Champloo-s01e01>: Unable to parse a valid provider from guid com.plexapp.agents.hama://anidb-1543/1/1?lang=en

Hope this helps

glensc commented 3 years ago

Seems the examples all have only one guid, and in form of anidb-$number, so need to add translation of anidb to providers that trakt support

glensc commented 3 years ago

Is there a library for anidb? Is there api documentation? first one to answer gets a 👍🏿 :D

simonc56 commented 3 years ago

need to add translation of anidb to providers that trakt support

If such translation exists, the anidb would be useless.

@eduardlleshi One Punch Man is on TMDB. Can't you move to TMDB ? How many of your shows are only on anidb ? For your anime not on TMDB, you could subscribe and add them.

eduardlleshi commented 3 years ago

@glensc do you still want me to make a PR for HAMA/ASS install instructions?

Currently, I do not have any Movies using the HAMA agent. Appreciate @Frazzer951 for jumping in and sending over some inspect examples!

@twolaw I have 8 anime shows, where 5 are on TVDB and 3 on AniDB:

ERROR: <Show:1353:Demon-Slayer:-Kimets>: Unable to parse a valid provider from guid com.plexapp.agents.hama://tvdb-348545?lang=en
ERROR: <Show:210:Dr.-Stone>: Unable to parse a valid provider from guid com.plexapp.agents.hama://tvdb-355774?lang=en
ERROR: <Show:839:My-Hero-Academia>: Unable to parse a valid provider from guid com.plexapp.agents.hama://tvdb-305074?lang=en
ERROR: <Show:560:One-Piece>: Unable to parse a valid provider from guid com.plexapp.agents.hama://tvdb-81797?lang=en
ERROR: <Show:785:One-Punch-Man>: Unable to parse a valid provider from guid com.plexapp.agents.hama://anidb-11123?lang=en
ERROR: <Show:538:Re:ZERO--Starting-Li>: Unable to parse a valid provider from guid com.plexapp.agents.hama://tvdb-305089?lang=en
ERROR: <Show:771:Record-of-Ragnarok>: Unable to parse a valid provider from guid com.plexapp.agents.hama://anidb-15932?lang=en
ERROR: <Show:1324:Tokyo-Revengers>: Unable to parse a valid provider from guid com.plexapp.agents.hama://anidb-15601?lang=en

I do not know the reason why HAMA decides to go with AniDB over TVDB. On my HAMA agent settings, TVDB is set as the first choice, while AniDB is a fallback. I believe AniDB contains more data on a given Anime show like better season/arc grouping, season covers, music, English titles/descriptions, cast and so on. Something that caught my eye is that the AniDB shows are more recent, and it could be that the info on TVDB is missing at the time a show gets released. I can inspect HAMA logs on this if that's valuable information for you.

I believe @glensc is right on finding a translation from AniDB IDs to something that Trakt understands. At the end of the day, we'd want to be able to log these as watched on Trakt, while Plex can still use AniDB as the source of information for the show.

@glensc this seems to be the official AniDB HTTP API.

simonc56 commented 3 years ago

Those 8 anime shows are on TMDB. Can you use TMDB as first HAMA source choice ? What happen when you use Plex agent instead of HAMA ?

it could be that the info on TVDB is missing at the time a show gets released

TVDB is not recommanded as source, TMDB is recommanded by Plex and Trakt.

eduardlleshi commented 3 years ago

TLDR; Using Plex TV Agent worked better than HAMA for the sync. There is one show with absolute episode numbers on TMDB that is failing. Renaming the show episodes to work with TMDB results in a PlexTraktSync exception.

I first set TMDB as the first source on HAMA. Deleted show "Record of Ragnarok", emptied trash on Plex, and re-added the show. The right data was populated on Plex, however it seems like AniDB was chosen again based on the inspect:

Inspecting 1396: <hama:anidb-15932:<Show:1396:Record-of-Ragnarok>>
URL: https://app.plex.tv/desktop/#!/server/a0d150f5e2145848a01569dddb7ca33c37ffe16b/details?key=/library/metadata/1396
Media.Type: show
Media.Guid: 'com.plexapp.agents.hama://anidb-15932?lang=en'
Guids:
  Guid: com.plexapp.agents.hama://anidb-15932?lang=en, Id: anidb-15932, Provider: hama
Metadata: {'collected_at': '2021-06-22:T21:28:54.000Z', 'media_type': 'digital'}
ERROR: <Show:1396:Record-of-Ragnarok>: Unable to parse a valid provider from guid com.plexapp.agents.hama://anidb-15932?lang=en

I then changed the Anime library to use "Plex TV Agent". Deleted the show, emptied Plex trash, and re-added it. The show data populated on Plex, and PlexTraktSync worked fine on marking the episodes as watched from Trakt -> Plex. I can verify that marking from Plex -> Trakt also worked.

INFO: Marking as watched in Plex: <tmdb:2567671:<Episode:1412:Record-of-Ragnarok-s01e01>>
INFO: Marking as watched in Plex: <tmdb:3029603:<Episode:1413:Record-of-Ragnarok-s01e02>>
INFO: Marking as watched in Plex: <tmdb:3029604:<Episode:1414:Record-of-Ragnarok-s01e03>>
INFO: Marking as watched in Plex: <tmdb:3029605:<Episode:1415:Record-of-Ragnarok-s01e04>>
INFO: Marking as watched in Plex: <tmdb:3029606:<Episode:1416:Record-of-Ragnarok-s01e05>>
INFO: Marking as watched in Plex: <tmdb:3029607:<Episode:1417:Record-of-Ragnarok-s01e06>>
INFO: Marking as watched in Plex: <tmdb:3029608:<Episode:1418:Record-of-Ragnarok-s01e07>>
INFO: Marking as watched in Plex: <tmdb:3029609:<Episode:1419:Record-of-Ragnarok-s01e08>>

Finally, moved all Anime shows to a different directory on the server, kept "Plex TV Series" as the Anime agent. All shows except 1 were matched against TMDB and data was being synced Plex <-> Trakt.

The only show I had problems with was "One Piece. Plex was not able to match the episodes using TMDB. The first episode of Season 21 is listed as S21E892 on TMDB, while on my files it's named as S21E01. I believe ASS (Absolute Series Scanner) was able to handle both absolute and seasoned episodes. These are the results of first episode of season 21 of One Piece:

File named as S21E01 - no metadata found from Plex. PlexTraktSync inspect shows:

PlexTraktSync inspect [0.15.0]
Inspecting 1896: <local:1896:<Episode:1896:One-Piece-s21e01>>
URL: https://app.plex.tv/desktop/#!/server/a0d150f5e2145848a01569dddb7ca33c37ffe16b/details?key=/library/metadata/1896
Media.Type: episode
Media.Guid: 'local://1896'
Audio: 'stereo', '日本語 (AAC Stereo)'
Video: 'h264'
Guids:
  Guid: local://1896, Id: 1896, Provider: local
Metadata: {'collected_at': '2021-10-18:T10:51:31.000Z', 'media_type': 'digital', 'resolution': 'hd_1080p', 'audio': 'aac', 'audio_channels': '2.0'}
WARNING: <Episode:1896:One-Piece-s21e01>: Skipping guid local://1896 because provider local has no external Id

File renamed to S21E892 - metadata matched against TMDB on Plex. The sync doesn't work still though:

Inspecting 1898: <tmdb:1817016:<Episode:1898:One-Piece-s21e892>>
URL: https://app.plex.tv/desktop/#!/server/a0d150f5e2145848a01569dddb7ca33c37ffe16b/details?key=/library/metadata/1898
Media.Type: episode
Media.Guid: 'plex://episode/5e16215a1c6e880037a08221'
Media.Guids: [<Guid:imdb://tt10544156>, <Guid:tmdb://1817016>, <Guid:tvdb://7228129>]
Audio: 'stereo', '日本語 (AAC Stereo)'
Video: 'h264'
Guids:
  Guid: tmdb://1817016, Id: 1817016, Provider: tmdb
  Guid: tvdb://7228129, Id: 7228129, Provider: tvdb
  Guid: imdb://tt10544156, Id: tt10544156, Provider: imdb
Metadata: {'collected_at': '2021-10-18:T10:51:31.000Z', 'media_type': 'digital', 'resolution': 'hd_1080p', 'audio': 'aac', 'audio_channels': '2.0'}
WARNING: <Episode:1898:One-Piece-s21e892>: Skipping guid tmdb://1817016 not found on Trakt
Trakt: https://trakt.tv/episodes/3567742
Watched on Plex: True
Watched on Trakt: False
Play history:
- 2021-10-18 11:21:53 by eduardlleshi with MacBook-Pro-2.local on osx

I ran docker-compose run --rm plex-trakt-sync to try and sync One Piece with the episode change (S01E892). An exception occurred on v0.15.0:

WARNING: <Episode:1898:One-Piece-s21e892>: Skipping guid tmdb://1817016 not found on Trakt
WARNING: Retry using search for specific Plex Episode
INFO: Marking as watched in Trakt: <tmdb:1817016:<Episode:1898:One-Piece-s21e892>>
Processing Anime  44% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4/9  [ 0:00:21 < 0:00:15 , 0 it/s ]
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/local/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/app/plex_trakt_sync/__main__.py", line 16, in <module>
    cli()
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1137, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1062, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1646, in invoke
    super().invoke(ctx)
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 763, in invoke
    return __callback(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/app/plex_trakt_sync/cli.py", line 23, in cli
    sync()
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1137, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1062, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 763, in invoke
    return __callback(*args, **kwargs)
  File "/app/plex_trakt_sync/commands/sync.py", line 131, in sync
    sync_all(walker=w, trakt=trakt, plex=plex, dry_run=dry_run)
  File "/app/plex_trakt_sync/commands/sync.py", line 40, in sync_all
    runner.sync(walker, listutil, dry_run=dry_run)
  File "/app/plex_trakt_sync/sync.py", line 23, in sync
    self.sync_watched(episode, dry_run=dry_run)
  File "/app/plex_trakt_sync/sync.py", line 64, in sync_watched
    m.mark_watched_trakt()
  File "/app/plex_trakt_sync/media.py", line 81, in mark_watched_trakt
    self.trakt_api.mark_watched(self.trakt, self.plex.seen_date)
  File "/app/plex_trakt_sync/decorators/nocache.py", line 14, in inner
    return method(*args, **kwargs)
  File "/app/plex_trakt_sync/decorators/rate_limit.py", line 23, in wrapper
    return fn(*args, **kwargs)
  File "/app/plex_trakt_sync/decorators/time_limit.py", line 18, in wrapper
    return fn(*args, **kwargs)
  File "/app/plex_trakt_sync/trakt_api.py", line 187, in mark_watched
    self.watched_shows.add(m.show.trakt, m.season, m.number)
AttributeError: 'dict' object has no attribute 'trakt'
simonc56 commented 3 years ago

Ok so except for One Piece all is good with Plex agent. You can check on this comparison page and scroll to One Piece and see what a mess this show is 😄

The first episode of Season 21 is listed as S21E892 on TMDB

This looks strange because on TMDB season 21 have 109 episodes only. You shold keep it named S21E01 but remove local data so Plex can perform a full search.

About the last bug AttributeError: 'dict' object has no attribute 'trakt' it is known as #487 and on its way to be fixed.

Frazzer951 commented 3 years ago

Is there a library for anidb? Is there api documentation? first one to answer gets a 👍🏿 :D

Not sure if these are exactly what you need but this is what i found with a quick search

API Definitions: https://wiki.anidb.net/API https://wiki.anidb.net/HTTP_API_Definition

AniDB Software: https://anidb.net/software If you look under the Library section there seems to be a couple python libs, but they say unix only, and dont seem like they are activly worked on

eduardlleshi commented 3 years ago

This looks strange because on TMDB season 21 have 109 episodes only. You should keep it named S21E01 but remove local data so Plex can perform a full search.

I don't have any local data stored for this show, only the video files. Kept the names as S21E01... and Plex is not able to match any metadata with "Plex TV Agent", so no episode titles or anything like that. Renaming them with absolute episode numbers S21E892 (this is the 892nd One Piece episode) and it does pull the metadata from TMDB, but no trakt sync. I'm okay missing a few shows though.

I'll wait on the #487 fix and see if I'd be able to sync it with S21E892.

simonc56 commented 3 years ago

In Plex, did you try to change episode ordering for the One Piece folder :

image

eduardlleshi commented 3 years ago

@twolaw I wasn't aware of that option. Using "TheTVDB (Aired)" worked for me. It was able to pull the info from TMDB for every episode, thus making the sync work as well.

It seems like HAMA/ASS was not needed, and with your help, I got to use the default Plex agent and match everything. Do we still need to look into the AniDB matching?

simonc56 commented 3 years ago

I got to use the default Plex agent and match everything.

Great!

Do we still need to look into the AniDB matching?

In my opinion, it's not needed and too complicated. As you can see, Plex is now very good to scrape data with its new TV Series agent.