One Piece TVDB4 Specials misbehaving

TrafalgaT commented 4 years ago

I noticed recently that all specials past special 30 (S31 - One Piece: Episode of Sabo) get meta matched to the episode 31 instead of the special

One Piece [tvdb4-81797].scanner.log One Piece [tvdb4-81797].filelist.log

sven-7 commented 4 years ago

I made several updates in ScudLee where One Piece specials were off or messed up. Not sure if this will ultimately close your issue or not, but brought light to that at the very least. Keep an eye out for that commit.

ZeroQI commented 4 years ago

Please confirm, it should be resolved

TrafalgaT commented 4 years ago

Hi, so...no i tried refreshing the meta for s00 still as it used to be, i removed the folder and moved it elsewhere, readded it and its still being matched wrong

ZeroQI commented 4 years ago

I will need the scanner logs, and one special number with issues as i need to check the mapping file, anidb and thetvdb for it...

TrafalgaT commented 4 years ago

One Piece [tvdb4-81797].filelist.log One Piece [tvdb4-81797].scanner.log

any special past 30 so for example special 31, or the recent one 39 stampede (they all get matched as episodes instead of specials)

ZeroQI commented 4 years ago

I do not see any specials in either logs

Library: 'no valid X-Plex-Token.id', root: '/share/CACHEDEV2_DATA/TV Anime', path: 'One Piece [tvdb4-81797]/S00 - Specials', files: '0', dirs: '2'
Plex scan start: 2020-07-06 09:59:32,849064
plexignore_files: '[]', plexignore_dirs: '[]'

[folder] One Piece [tvdb4-81797]/S00 - Specials/[0.1] Arc Movies
[folder] One Piece [tvdb4-81797]/S00 - Specials/[0.2] Arc Special Episodes
[no files detected] Grouping folder skip as >1 folder [return]

No episode 39 can be found in either logs...

TrafalgaT commented 4 years ago

yes, i saw that also, is there a special reason it keeps skipping the folder ? the folder structure im using used to work aswell (grouping) but the funny thing is it matches everything up to s30, everythin past that is messed up and it keeps missing the s00 folder now whenever im trying to update the meta

ZeroQI commented 4 years ago

How can the metadata be wrong when you say "it keeps missing the s00 folder now " The second phrase in the readme states:

If a file is not showing in plex or showing at the wrong season and/or episode number, or is not passing through the forced id, then it is a scanner issue. Anything else is metadata related and agent specific.

I don't think you should have a folder per episode, especially not for specials, especially not in a grouping folder, not sure if tvdb4 mode doesn't add itself complexity there...

The scanner will not re-scan if the folder content didn't change... you might have to do the Plex Dance...

TrafalgaT commented 4 years ago

Ok I think you are not understanding what I am saying.

Initally I added the Series (with tvdb4-#) as a folder, grouped by specials and seasons, subgrouped by arcs. All of this worked flawlessly, until recently when i noticed that the specials were not being matched properly. Everything before special episode 31 got matched properly, everything after 31 is not getting matched. This is not a new process. I realised this when i added the latest special Stampede, which wrongly matched to epsiode 38 of the series. so to answer your intial accusation. the metadata was correctly matched for everything else, all the seasons are fine. This issue just concerns the specials folder and all episodes titled 30+ consecutively.

Update: im running the dance now, let see if that will fix it

ZeroQI commented 4 years ago

Thanks for the precision. The files are added to Plex but I cannot see them in scanner logs (which makes things tricky), but if you tell me the specials 31+ ends up in Plex with the exact same episode number and season 0, then we can skip the scanner logs, as you assign manually yourself the season number so the scanner is transparent on the season and episode number assignment

If that is the situation, then it is a metadata issue and we need to check the mapping so the agent logs would help to know if it uses the latest mapping files or a cached version It would also show which source provided the meta, and the dual way mapping used between ANiDB and TheTVDB

TheTVDB s00e31 https://thetvdb.com/series/one-piece/episodes/5279341 AniDB S00e21 https://anidb.net/episode/170966

  <anime anidbid="69" tvdbid="81797" defaulttvdbseason="a" episodeoffset="" tmdbid="" imdbid="">
    <name>One Piece</name>
    <mapping-list>
      <mapping anidbseason="0" tvdbseason="0">;1-28;2-3;3-9;4-10;5-14;6-0;7-0;8-0;9-0;10-0;11-0;12-0;13-0;14-0;15-0;16-32;17-23;18-24;19-29;20-30;21-31;22-0;23-33;24-35;25-0;26-37;27-38;</mapping>
    </mapping-list>
  </anime>

The mapping file is correct and your episode number should be s00e31 in Plex using TVDB4 mode I need the agent logs to SEE the metadata source used, and which provided wrong info and if the agent has loaded the right mapping from ScudLee's XML mapping files

TrafalgaT commented 4 years ago

ill try my best to find those files, right now i have to wait for the dance to finish, but if the problem presists ill dig into it. its kinda tricky finding the right logs on qnap for me because there are just so many different log folders and different individual log files.

ZeroQI commented 4 years ago

should be in agent data folders for HAMA agent

TrafalgaT commented 3 years ago

so:

1) I did the Plex Dance, unsuccessfully. Back to status Quo, all the metadata has been matched, but in case of the specials, it has reverted to the previous state. All episodes past special number 30 (S31 - Episode of Sabo), all specials get matched to the 'series' episode number E031+.

2) I have tried to locate the log,but

/Plex Media Server/Plug-in Support/Data/com.plexapp.agents.hama/DataItems/_Logs

but its empty so there are no logs in there

ZeroQI commented 3 years ago

Without logs, it's tricky to solve... The code was altered in both ASS and HAMA yesterday to not require the token anymore... is there folders with the library name in there ?

Can you confirm in Plex that for episode 31, it is showing in Plex as the same series and in season 0 at episode 31 ?

TrafalgaT commented 3 years ago

ok so this is what it looks like in plex

s00_view_plex_31plus

these correspond to the actual 'arc' episodes E31.....E38 and not the special numbers S31 to S38

and as for my library itself this is what the library folder looks like library

and the example for one piece

one_piece_example

EDIT

I could try to update both HAMA and ASS and get back to you, after a refresh One Piece [tvdb4-81797].filelist.log One Piece [tvdb4-81797].scanner.log root-share-CACHEDEV2_DATA-TV Anime.filelist.log root-share-CACHEDEV2_DATA-TV Anime.scanner.log _unknown_folder.agent-update.log

ZeroQI commented 3 years ago

Logs shows no episode present physically apart from s00e39 "/share/CACHEDEV2_DATA/TV Anime/One Piece [tvdb4-81797]/S00 - Specials/[0.1] Arc Movies/One Piece - 39 - Stampede.mkv"

I do not see the scanner logs about adding specials for some reason... you do use transparent folders but it seem to skip so shouldn't be showing in plex...

=================================================================================
============================================================================
Library: 'TV Anime', root: '/share/CACHEDEV2_DATA/TV Anime', path: 'One Piece [tvdb4-81797]/S00 - Specials', files: '0', dirs: '2'
Plex scan start: 2020-07-07 10:05:42,894914
plexignore_files: '[]', plexignore_dirs: '[]'
=============================================================================================================================================================
[folder] One Piece [tvdb4-81797]/S00 - Specials/[0.1] Arc Movies
[folder] One Piece [tvdb4-81797]/S00 - Specials/[0.2] Arc Special Episodes
[no files detected] Grouping folder skip as >1 folder [return]
=============================================================================================================================================================

I am confused...

TrafalgaT commented 3 years ago

you are not the only one who is confused..... they show up as Specials (example attached) as they show, up to special 30, like i mentioned, everything after that s31+....well its been mentioned before

unnamed

meisnate12 commented 3 years ago

I have the same setup using tvdb4 and I'm having the same problem except that for me only the first 8 specials work, which is the same number of episodes that I have in Season 1 or the show. I added all of Season 2 to Season1 to total 30 episodes in Season 1 and then removed one piece did the plex dance and readded it. After which the first 30 episodes in the specials all had the right names. It seems like however many episodes that are in season 1 lets the same number of specials get the right meta data but after that for some reason it takes the meta data from the next series episode.

meisnate12 commented 3 years ago

Here are my logs if it helps any

One Piece [tvdb4-81797].agent-search.log One Piece [tvdb4-81797].agent-update.log One Piece [tvdb4-81797].filelist.log One Piece [tvdb4-81797].scanner.log

ZeroQI commented 3 years ago

Well spotted! indeed the data gets overwritten by the ep metadata!

Let's use impacted s00e34 ONE PIECE FILM GOLD showing "Everyone's Gathered! Usopp Speaks the Truth About Nami!" Metadata at the end of thetvdb part is already wrong '34': {'absolute_index': 34, 'content_rating': 'TV-Y7', 'duration': 1500000, 'language_rank': 0, 'originally_available_at': '2000-07-26', 'rating': 7.8, 'summary': 'A fierce argument over where Nami\'s loyalties lie divides the Straw Hat Pirates and only one thing\'s for sure - no one is willing to risk their life for a traitor.', 'thumbs': {'https://thetvdb.plexapp.com/banners/episodes/81797/5540965.jpg': ('TheTVDB/episodes/5540965.jpg', 1, None), 'https://thetvdb.plexapp.com/banners/episodes/81797/361923.jpg': ('TheTVDB/episodes/361923.jpg', 1, None)}, 'title': 'Everyone\'s Gathered! Usopp Speaks the Truth About Nami!'},

[X]   s0e34 s00e034 anidbid:         air_date: 2016-07-23 abs_number: 0, title: One Piece Film Gold
 - [1] title:   [en] One Piece Film Gold
common.LoadFile() - File cached locally - Filename: 'TheTVDB/json/81797/episode_5540965_ja.json', Age: '0.51 days', Limit: '6 days', url: 'https://api.thetvdb.com/episodes/5540965'
 - [2] summary: [ja] 新世界の“怪物”が動き出す!!!物語の舞台は、世界中の名立たる海賊、海兵、大富豪たちが集う、世界最大のエンターテインメントシティ“グラン・テゾーロ”。そこでルフィたちの前に現れたのは、金の力で世界政府ですら容易く動かし、天竜人をも懐柔する絶対的な権力を持つ“黄金帝”ギルド・テゾーロ。豪華絢爛な街の裏ではテゾーロによる圧倒的な支配が渦巻き、ルフィたちはなすすべもなく街の非情なルールに飲み込まれていく。そんな中、テゾーロから天竜人への「史上最高額の天上金」の存在が明らかとなり、その支配力は世界を揺るがすほどの暴走をはじめていた。遂に動き出した新世界の“怪物”に、ざわめきはじめる世界政府、そして革命軍――。「自由とは、支配とは」この海を制するための答えを求め、ルフィたちの信念を懸けた闘いが幕を開けようとしていた。
 - [ ] thumb: https://thetvdb.plexapp.com/banners/episodes/81797/5540965.jpg

Meta is ok but over-written by:

[X]    s3e4 s00e034 anidbid:         air_date: 2000-07-26 abs_number: 34, title: Everyone's Gathered! Usopp Speaks the Truth About Nami!
 - [1] title:   [en] Everyone's Gathered! Usopp Speaks the Truth About Nami!
 - [1] summary: [en] A fierce argument over where Nami's loyalties lie divides the Straw Hat Pirates and only one thing's for sure - no one is willing to risk their life for a traitor.
common.LoadFile() - File cached locally - Filename: 'TheTVDB/json/81797/episode_361923_ja.json', Age: '1.46 days', Limit: '6 days', url: 'https://api.thetvdb.com/episodes/361923'
 - [ ] thumb: https://thetvdb.plexapp.com/banners/episodes/81797/361923.jpg

The

https://github.com/ZeroQI/Hama.bundle/blob/master/Contents/Code/TheTVDBv2.py

numbering ok line 146

season, ep wrong in 167-175 (tvdb4)

  elif metadata_source in ('tvdb3', 'tvdb4'):  
    if str(abs_number) in list_abs_eps:  #if abs id exists on disk, leave specials with no translation
      for s in media.seasons:
        if str(abs_number) in media.seasons[s].episodes:
          season, episode = s, str(abs_number)
          break
      else:  continue

if abs number on disk, we pick the first season that have the abs_number and season becomes this season, episode becomes the abs number So s003e4 abs number 34 should become s003e34 (or others if assigned manually) but not s00e34 shall we skip if already exist? Please test replacing line 167-173 with:

  elif metadata_source in ('tvdb3', 'tvdb4'):  
    if str(abs_number) in list_abs_eps:  #if abs id exists on disk, leave specials with no translation
      for s in media.seasons:
        if str(abs_number) in media.seasons[s].episodes and not Dict(TheTVDB_dict, 'seasons', s, 'episodes', str(abs_number)):
          season, episode = s, str(abs_number)
          break
      else:  continue

TrafalgaT commented 3 years ago

tried it and the meta did not change for the affected specials

meisnate12 commented 3 years ago

so i tried you fix and its better but still has a small bug. All the special episodes that have metadata on TVDB now work fine but i have 5 more videos i include in the specials that dont have metadata on TVDB and those still picked up the metadata for episodes 40-45 where they shouldn't have had any metadata

ZeroQI commented 3 years ago

Give me your code update and the update log and I should be able to fix...

meisnate12 commented 3 years ago

TheTVDBv2.txt One Piece [tvdb4-81797].agent-update.log

TrafalgaT commented 3 years ago

so i tried you fix and its better but still has a small bug. All the special episodes that have metadata on TVDB now work fine but i have 5 more videos i include in the specials that dont have metadata on TVDB and those still picked up the metadata for episodes 40-45 where they shouldn't have had any metadata

i do not know what you did, i did the same edit to the thetvdbv2.py file and for me it is still like it used to be, still missmatched specials

meisnate12 commented 3 years ago

i removed one piece from my library then cleaned my bundles and emptied my trash i restarted plex after that then one it was back up i readded one piece

TrafalgaT commented 3 years ago

Ok things are getting weird at this stage. All the mismatched episode have now correctly matched to the respective special meta.....the only thing is the thumbnails are still the “episode” tvdb thumbnail and not from the special (all of this is with the latest advises line changes to thetvdb2.py file that @ZeroQI mentioned above

ZeroQI commented 3 years ago

@TrafalgaT You might need to check the thumbs in plex as the previous screenshot will sadly be the main one but a second one should be loaded if the bundles weren't cleared... Logs could show which thumb was provided by the agent because there could be some plex issue there...

ZeroQI commented 3 years ago

@meisnate12 opened log, it seems ep 40-44 are impacted and translated wrongly to season 0:

[X]   s3e10 s00e040 anidbid:         air_date: 2000-09-06 abs_number: 40, title: Proud Warriors! Sanji and Usopp's Fierce Battles!
[X]   s3e11 s00e041 anidbid:         air_date: 2000-09-13 abs_number: 41, title: Luffy at Full Power! Nami's Determination and the Straw Hat!
[X]   s3e12 s00e042 anidbid:         air_date: 2000-09-27 abs_number: 42, title: Explosion! Fishman Arlong's Fierce Assault from the Sea!
[X]   s3e13 s00e043 anidbid:         air_date: 2000-09-27 abs_number: 43, title: End of the Fishman Empire! Nami's My Friend!
[X]   s3e14 s00e044 anidbid:         air_date: 2000-10-11 abs_number: 44, title: Setting Out With a Smile! Farewell, Hometown Cocoyashi Village!

These episodes should not be getting metadata

According TheTVDB specials are s00e01-39: https://thetvdb.com/series/one-piece/seasons/official/0 so your specials are misnumbered, sould be in "Extras" folder not showing under Plex ideally or correctly numbered
Issue: anybody with a special with no metadata at the same ep number as an absolute number episode, the special will have the absolutely numbered ep metadata [bad]

Fix: if a special has an absolute episode, then it is mapped to season 1+, there is never a need for custom mapping to season 0. Please replace previous quoted code with shorter replacement and please report if working as expected but i think i got it right this time:

  elif metadata_source in ('tvdb3', 'tvdb4'):  
    for s in media.seasons:  #if abs id exists on disk, leave specials with no translation
      if  str(abs_number) in list_abs_eps and str(abs_number) in media.seasons[s].episodes and s!="0":
        season, episode = s, str(abs_number)
        break

meisnate12 commented 3 years ago

yup that last code change fixed it completely for me

ZeroQI / Hama.bundle

One Piece TVDB4 Specials misbehaving #405