ZeroQI / Absolute-Series-Scanner

Seasons, absolute mode, Subfolders...
1.02k stars 155 forks source link

Movies in a series library and anidb2 force id mode assign series "xxx [tvdb-1]" #302

Closed OmegaToken closed 4 years ago

OmegaToken commented 4 years ago

I tried to order my collection For that I renamed all movies and episodes so that the containing folder holds the [anidb2-????] Tag.

Most of the files are working but some files got placed in the xxx [tvdb-1] folder. The folder look like a collection of errors. There are different Movies and Episodes mixed into one entry as versions. (See Image) Screenshot 2020-09-21 170957

The log files do give me no hint, or at least I do not know what I am looking for.

I use the actual ASS and HAMA Plugins

ZeroQI commented 4 years ago

The scanner change the id from anidb2 to thetvdb as per the SCANNER design, this is not an agent issue...

Why do you believe the movies would exist in TheTVDB which is a Series database? There is no tvdbid in mapping file... https://raw.githubusercontent.com/ScudLee/anime-lists/master/anime-list-master.xml

  <anime anidbid="8832" tvdbid="movie" defaulttvdbseason="1" episodeoffset="" tmdbid="" imdbid="tt2140203">
    <name>Ookami Kodomo no Ame to Yuki</name>
  </anime>
  <anime anidbid="8289" tvdbid="movie" defaulttvdbseason="1" episodeoffset="" tmdbid="" imdbid="tt2061702">
    <name>Hotarubi no Mori e</name>
  </anime>
  <anime anidbid="2530" tvdbid="movie" defaulttvdbseason="1" episodeoffset="" tmdbid="" imdbid="tt0104740">
    <name>Little Nemo</name>
  </anime>

Bayonetta should work for anidb2

  <anime anidbid="10114" tvdbid="357271" defaulttvdbseason="1" episodeoffset="" tmdbid="" imdbid="tt3306776">
    <name>Bayonetta: Bloody Fate</name>
  </anime>

My recommendations:

ZeroQI commented 4 years ago

Please use anidb OR tvdb mode only, as 3 of the anidb2 mode you chose are devoid of any logic You cannot map a single movie to a non-existant series. What is wrong with [anidb-xxx] force id?

OmegaToken commented 4 years ago

@ZeroQI

What is wrong with [anidb-xxx] force id?

Based on the description I got the impression that if using anidb-xxx Episodes of Series will not combined into one entry.

ZeroQI commented 4 years ago

It needs the tvdbid in mapping file and mapping matching tvdb numbering exactly if using anidb2

You can combine them yourself by numbering as per thetvdb.... They look like standalone movies without tvdb series or TheTVDB do not consider them as a special of an existing series

With what tvdbid series were you trying to group little nemo? The scanner should have not remapped to be fair, but at the same time it is not a standard and easy mode you put on by default.

OmegaToken commented 4 years ago

I just downloaded ASS and Hama and created the library after scanning the description for important settings. I did not make any modification to mapping files or whatsoever. The only thing I tried is to add AniDB in the Settings for Posters. So I'm pretty new and do not realy have a clue...yet

I also did a test with a few Series and Movies [anidb-xxx]. The problem with the "xxx [tvdb-1]" still remains.

I did not group anything by myself or did try. I just want plex to recognize all Movies/Series based on there anidb id and display it. The Season grouping inside a series with the werging of Sequels is a bonus that would be great.

OmegaToken commented 4 years ago

@ZeroQI I checked the mapping and did a minimalistic test run and there seems to be some problems. I droped the old one did make a cleanup and then added a new mediatek. As you can see on the images AnimeFolder Plex .hack is not recognised at all. Whitch is I suspect because of the leading . whitch on linux systems would mean hidden. -> confirmed after renaming the folder

1000-nen Joou is not recogliced at all and can not be found in the mediatek. Ookami Kodoo no Ame to Yuki did show up in the xxx folder.

All Series/Movies listed have an valid ScudLee tvdbid mapping. The only exception is Ookami Kodoo where the mapping has no id but the entry movie.

Ookami Kodomo no Ame to Yuki

Copy From ASS description

Tha Last Straw: After renaming some folders manually (just addind a letter in the folder name somewhere). The Movies and series started to be shown correctly so I asume this is some strange plex database caching problem?

However I did put way to mutch time in this

ZeroQI commented 4 years ago

Notes

You need to list th issues clearly and provide logs. Here is a break-down of your last post:

OmegaToken commented 4 years ago

@ZeroQI Since I suspect some plex database corruption I deleted the database an will rebuild it with my normal movies and series. After that I will delete all the logfiles start plex and create the mediatek Anime with some Testfiles again and send you all created Logfiles, if my problem still remains?

Should I try anidb2-xxx again or anidb-xxx?

As for Ookami Kodomo no Ame to Yuki I did copy the exact entry into my comment, It only seems that github did hide the xml since I did not encapsulate it in an code tag. Sorry my bad.

OmegaToken commented 4 years ago

So ok Fresh from the start.... after creating andloading my test files. My test environment AnimeFolder The resulting logs

root.scanner.log Plex Media Scanner.log Plex Media Server.log

Current Status .hack is not shown at all 1000-nen Joou is not shown at all but it is mixed up as version in the xxx [tvdb-1] folder. Naruto looks ok. Naruto Narutimate Hero 3 looks ok Naruto Shippuden looks ok Ookami Kodomo moved to xxx [tvdb-1] folder Nauto The Cross Roads only japanese Title is shown. -> This is new, an hour ago this was working. It is realy hard to track anything down if the result is different every time I run the test. Even with same starting parameters.

As for tvdb Ookami has is the only title in this test that has no tvdbid. The entry in the ScrudLee xml is posted above.

OmegaToken commented 4 years ago

Just to be clear the .hack is not an issue since I simply can rename it to dothack and it is working. The other things are...

However renaming .hack shows the next problem. Plex now shows me the wrong poster images. Screenshot 2020-09-22 163923

Does PLEX, ASS or HAMA work in any way as expected and as it should?

ZeroQI commented 4 years ago

Issues mentioned

There is no series specific agent or scanner logs attached... What other logs did you have alongside the file "root.scanner.log" in the same folder ?

OmegaToken commented 4 years ago

There are none. Do I have to activate them somewhere?

Also I did delete my whole PLEX Database directory to ensure there is no left over metadata.

OmegaToken commented 4 years ago

Ok I think I found something. 1.) The Logs for the Plugins are not located in the Logs folder instead they can be found in the Plug-in Support\Data\com.plexapp.agents.hama\DataItems_Logs folder 2.) There is an bug in the agent where the anidb hint is ignored. For Example: 1000-nen Joou (1982) [anidb-764].agent-search.log As you can see the anidb hint is ignored instead an tvdb hint with id 1 is enforced, this is compleatly wrong. Well the scanner log for the folder gives away an hint

=============================================================================================================================================================
Forced ID (series folder) - source: "anidb", id: "764"
-------------------------------------------------------------------------------------------------------------------------------------------------------------
misc_count: {'nen': 2, 'Complete': 1, 'Movie': 1, 'T1': 1, '1': 1, '1000': 2, 'Trailer': 1, 'Joou': 2}
misc_words: []
-------------------------------------------------------------------------------------------------------------------------------------------------------------
"1" s01e001                         "Word Search" "" "1000-nen Joou - 1 - Complete Movie [Henshin-BOX](41AC7FC1).mkv"
"1000-nen Joou (1982) [anidb-764]" s00e201                         "ANIDB_RX-3" "Trailer" "1000-nen Joou - T1 - Trailer [Henshin](CAC4AA38).mkv"
=============================================================================================================================================================

The "1" did give me an idea and I changed the Filename Now the scanner.log shows

=============================================================================================================================================================
Forced ID (series folder) - source: "anidb", id: "764"
-------------------------------------------------------------------------------------------------------------------------------------------------------------
misc_count: {'nen': 2, 'Complete': 1, 'Movie': 1, 'T1': 1, '1000': 2, 'Trailer': 1, 'Joou': 2}
misc_words: []
-------------------------------------------------------------------------------------------------------------------------------------------------------------
"1000-nen Joou (1982) [anidb-764]" s00e201                         "ANIDB_RX-3" "Trailer" "1000-nen Joou - T1 - Trailer [Henshin](CAC4AA38)[anidb-764].mkv"
"1000-nen Joou (1982) [anidb-764]" s00e501                         "__" "Complete Movie" "1000-nen Joou - Complete Movie [Henshin-BOX](41AC7FC1)[anidb-764].mkv"
=============================================================================================================================================================

And the agent search log changed to 1000-nen Joou (1982) [anidb-764].agent-search.log

As you can see now Title changed from "1" to "1000-nen Joou (1982) [anidb-764]" and name changed from "None" to "Complete Movie" and most important the Forced ID log now shows source: anidb and id 764 what was expected in the first place. The fact that a simple number in the title throw the forced id of balance is troublesome since that should never be the case as soon a force id is present.

After that I tried to rename the file to "1000-nen Joou - 2 - Complete Movie Henshin-BOX.mkv" and guess what now the message reads

Forced ID - source: tvdb, id: 2, title: 'xxx'

So there is a bug where a number in the title overrides the forced id and the source with tvbd and whatever number is found.

ZeroQI commented 4 years ago

"D:\Anime\1000-nen Joou (1982) [anidb-764]\1000-nen Joou - 1 - Complete Movie Henshin-BOX.mkv" indeed should keep the forced id, but "- Complete movie" was meant to be single file if a folder created by WebAOM, and all before " - Complete Movie" is the series title, which should have been "1000-nen Joou - 1"... The title is wrongly changed to "1" (erasing forced id), and because the title has been changed to a number, the agent considers it a tvdb id...

https://github.com/ZeroQI/Absolute-Series-Scanner/blob/master/Scanners/Series/Absolute%20Series%20Scanner.py line 954 if " - Complete Movie" in ep: ep, title, show = "01", ep.split(" - Complete Movie")[0], ep.split(" - Complete Movie")[0] ### Movies ### If using WebAOM (anidb rename) To test if it resolves: if " - Complete Movie" in ep: ep = "01" ### Movies ### If using WebAOM (anidb rename)

OmegaToken commented 4 years ago

@ZeroQI Thanks for the help. I think I'm getting better at understandig Plex and scripts at all. Damn I would kill if there was a posibility to debug the python scripts.

"D:\Anime\1000-nen Joou (1982) [anidb-764]\1000-nen Joou - 1 - Complete Movie Henshin-BOX.mkv" indeed should keep the forced id, but "- Complete movie" was meant to be single file if a folder created by WebAOM, and all before " - Complete Movie" is the series title, which should have been "1000-nen Joou - 1"... The title is wrongly changed to "1" (erasing forced id), and because the title has been changed to a number, the agent considers it a tvdb id...

I do see the point, the problem is that many Movies in AniDB are called "Complete Movie". Especially if there are "Part 1 of x" Versions of the movie too, in that case the name would be "Part 1 of x"

https://github.com/ZeroQI/Absolute-Series-Scanner/blob/master/Scanners/Series/Absolute%20Series%20Scanner.py line 954 if " - Complete Movie" in ep: ep, title, show = "01", ep.split(" - Complete Movie")[0], ep.split(" - Complete Movie")[0] ### Movies ### If using WebAOM (anidb rename) To test if it resolves: if " - Complete Movie" in ep: ep = "01" ### Movies ### If using WebAOM (anidb rename)

The change does work at least as far as I have tested it. I will start to scan my whole library and the see what the out come is. If the change does do more harm than could I could change the naming template for all files in the type movie. The only problem with that is that AniDB nows some types (Others, Web, ...) that can hold movieish or serieish named files, in that case it would be an case to case naming which I would like to avoid.

Conclusion:

[ASS] .hack is not shown at all due to leading dot. That is a bug. ASS scanner logs missing

They did not show up in any log so I assume Plex is ignoring them and there is nothing agents or scanners can do about it

[ASS] Series showing under "xxx [tvdb-1]" ("1000-nen Joou" and "Ookami Kodomo"). ASS scanner logs missing

The suggested change seems to solve this particular problem

[HAMA] "Naruto The Cross Roads" only japanese Title is shown. The answer is in the Hama series specific logs. You may be AiDB banned temporarily

This was my fault I messed up the language priority and this particular movie did only have a few languages inserted. After fixing it the english Title was shown as expected.

[HAMA] same poster on multiple series. The answer is in the Hama series specific logs. Since the poster is from TheTVDB, and they all point to the same series, if there is not enough posters it is possible, or you didn't "clear bundles" and Plex remembers the database information as it was the same files, even if you deleted the libraries...

Hard to tell if it is a bug or just work as designed. You are right that they all point to the same Series, however I changed posters to be taken from anidb only. My expectation was that then the one Poster that is visible on the anidb page for that series/movie is used. That does not seem to be the case since even after recreating the library more that one poster was available even when AniDB only should deliver one. I can rule out caching since I deleted all caches and even used a new sourcefolder and a new library to test this.

Suggestion: Since taking the information from the Filename seems quite fragile. Wouldn't it be better if on folders decorated with [anidb-xxx] all information regarding the servies/movie is only taken from anidb. And thinking ahead would'nt it be better to let files be decorated with [anidb-yyy] where yyy is the episodeid from anidb and regardles of file name every data is taken from anidb too. If hints [anidb|tvdb|...] are given filenames and directory names should be ignored completly. That way regardles of file/directory name the entries get always the expected Entry.

For completition I use AniAdd to rename and organize my Anime Library The actual script is posted below. I take suggestions on that site too

Trunc(str, len):= $repl(%str%, ".{" $len($repl(%str%, "(.?){" %len% "}$", "")) "}$", "")
TruncEllipse(str, len):= { $len(%str%) = $len($Trunc(%str%, %len%)) ? %str% : $Trunc(%str%, %len%) "..." }
TooLong(path):= { $max($len(%path%), "256") = "256" ? "0" : "1" }

AT:=[%ATr%,%ATe%,%ATk%]
AT:= $repl(%AT%,'^\.',"dot")
AT:= $repl(%AT%,'[\\\":/*|<>?]',"")
AT:= $repl(%AT%,' \([0-9][0-9][0-9][0-9]\)$',"")
ET:=[%ETe%,%ETr%,%ETk%]
GT:=" [" [%GTs%,%GTl%] "]"
GT:=$repl(%GT%," \[raw\]","")
GT:=$repl(%GT%," \[\]", "")
CRC:="("$uc(%FCrc%)")"
NaN:= $repl(%EpNo%, "[0-9]", "_")

EpNoPad:= { $substr(%NaN%, "0", "1") = "_" ? $pad(%EpNo%,$max($len(%EpHiNo%),$len(%EpCount%)),"0") : %EpNo% }

Ver:={%Ver%="1"?"":"v"%Ver%} 

FileNameLong:=%AT%" - "%EpNoPad% %Ver%" - "%ET%%GT%%CRC%
FileNameMedium:=$TruncEllipse(%AT%,"79")" - "%EpNoPad% %Ver%" - "$TruncEllipse(%ET%, "17")%GT%%CRC%
FileNameShort:=$TruncEllipse(%AT%,"97")" - "%EpNoPad% %Ver%%GT%%CRC%
PathNameLong:="D:\Anime\" %AT%" ("%AYearBegin%") [anidb-"%AId%"]"
PathNameShort:="D:\Anime\" $TruncEllipse(%AT%,"76")" ("%AYearBegin%") [anidb-"%AId%"]"

FullPathLong:=%PathNameLong% "\" %FileNameLong%
FullPathMedium:=%PathNameLong% "\" %FileNameMedium%
FullPathShort:=%PathNameLong% "\" %FileNameShort%
FullPathShort2:=%PathNameShort% "\" %FileNameMedium%

FileName:={$TooLong(%FullPathLong%)="1" ? {$TooLong(%FullPathMedium%)="1" ? {$TooLong(%FullPathShort%)="1" ? {$TooLong(%FullPathShort2%)="1" ? %FileNameShort% : %FileNameMedium%} : %FileNameShort%} : %FileNameMedium%} : %FileNameLong%}
PathName:={$TooLong(%FullPathLong%)="1" ? {$TooLong(%FullPathMedium%)="1" ? {$TooLong(%FullPathShort%)="1" ? %PathNameShort% : %PathNameLong%} : %PathNameLong%} : %PathNameLong%}
ZeroQI commented 4 years ago

Wouldn't it be better if on folders decorated with [anidb-xxx] all information regarding the servies/movie is only taken from anidb

  • Translation: movies in series libraries cause a scanner issue, let's change the agent???
  • It will use specials numbering from the source given but cross reference anidb eps to TheTVDB. TheTVDB has better episode title languages also.
  • you would get crap quality posters, no episode titles nor episode summaries...
  • We cannot make a distinction between assigned an id and forced assigned the same id

"Hard to tell if it is a bug or just work as designed. "

You have to understand the difference between scanner and agent.

Please confirm if the code update works for you and i will include in master code

OmegaToken commented 4 years ago

I can confirm that the problem with the xxx folder is gone.