Absolute numbering with specials mapping issue impacting One Piece TVDB4

[crazygambit]

I know this has been brought up before, but there hasn't been an acceptable solution yet for One Piece with TVDB4. The culprit is episode 590, which was a special.

https://github.com/ZeroQI/Hama.bundle/issues/264

Here the proposal was to use a folder with an offset, but that messes up with the seasons' cut off episodes (this is particularly important now that a new arc is starting). Renaming episodes is a no go. Sonarr is getting the correct episodes, having to rename them "wrong" just to get them to play nice with HAMA seems like the wrong way to go.

https://github.com/ZeroQI/Absolute-Series-Scanner/issues/194

Here some solutions were proposed, but I don't think anything came out from them. Given One Piece is hugely popular and the show used as an example for TVDB4 in the readme, it's kind of a bad look that it doesn't work. And it's so close to being perfect too! The idea to use arcs as seasons is absolutely fantastic, much better than the mess that is the TVDB order and something that to my knowledge no other Agent can do, on any platform. I think it's worth revisiting this, particularly now that after a couple of years a new arc is starting, and a lots of people might realize they have the wrong seasons (like I did, I didn't even notice the episodes were wrong before Plex mistakenly started the new season ahead of time).

TLDR: TVDB4 is AWESOME, but there's a bug on One Piece a hugely popular show and the example used on the readme, so it might be worth to fix it.

[sven-7]

@architdate -- I'll try tvdb3 today.

@ZeroQI - Okay, that makes sense. I'll try that today. What about 892? What are you seeing there?

[ZeroQI]

892: https://thetvdb.plexapp.com/banners/episodes/81797/7228129.jpg https://www.thetvdb.com/series/one-piece/seasons/21 Screenshot match. If the wrong screenshot is used by the agent, and the error is corrected, the wrong one is still the default...

[architdate]

Plex dance did the job, however now TVDB3 doesnt work for me and actually follows TVDB numbering as opposed to local directory numbering. So I made the shift to TVDB4 and despite having absolute numbered episodes in seasonal folders, it is still picking up the mapping from the cached tvdb4.mapping.xml. Is this the expected behaviour? (im assuming not) and if not, could you let me know what I am doing wrong?

@ZeroQI Thanks!

[sven-7]

Plex Dance also did the trick for me on TVDB4.

I need commit a new tvdb.mapping once this code gets committed to HAMA.

If you want, you can make file called tvdb4.mapping in your One Piece folder with the following contained within it:

01|001|030|East Blue - Gathering the Crew
02|031|044|Arlong Park
03|045|063|Entering into the Grand Line 
04|064|077|Whiskey Peak and Little Garden
05|078|092|Drum Island
06|093|135|Alabasta
07|136|152|Jaya Arc
08|153|195|Skypiea
09|196|206|The Naval Fortress G8
10|207|228|Davy Back Games - The Foxy Pirate Crew
11|229|263|Water Seven
12|264|326|Enies Lobby
13|327|384|Thriller Bark
14|385|407|Sabaody Archipelago
15|408|421|Island of Women / Amazon Lily
16|422|456|Impel Down
17|457|489|Marineford
18|490|516|Post-War Arc
19|517|574|Fishman Island
20|575|628|Punk Hazard
21|629|746|Dressrosa
22|747|782|Zou Arc
23|783|877|Whole Cake Island Arc
24|878|891|Reverie Arc
25|892|892|Wano Arc (unknown length)

[architdate]

@sven-7 so this tvdb4.mapping will overwrite the cached tvdb4.mapping.xml file in the ASS cache folder? However, I thought that TVDB4 checks for seasonal folders too? This is fine as a temporary solution though!

[sven-7]

Yep - once this code gets submitted, I'll update the tvdb4.mapping.xml in ASS so that it will pull down for everyone. That being said, you can use this local file to make personal modifications if you want.

I don't think tvdb4 does - but I've never tried that. @ZeroQI would know best.

[architdate]

@sven-7 Thanks for the help!, but I don't think the mapping worked from season 2 onwards (screenshots attached). Is there anything that was done wrong? Season 1 got named perfectly btw! (the custom mapping is wikipedias mapping)

[sven-7]

I'm not sure how the folders are going to affect this -- I have all of my One Piece files in one folder. Do you have the tvdb4.mapping file in the top level One Piece folder?

[architdate]

Yeap its in the root folder of the anime (so in D:/Media/Anime/One Piece)

[architdate]

Also i read the README.md in ASS, and it states for tvdb4: Will take the arc definitions from tvdb4.mapping.xml and posters from tvdb4.posters.xml unless the absolute numbered episodes were placed in season folders already

Which is why i thought that it should have been seasonally sorted even without the mapping file. (but even with the mapping file, it seems to be having issues)

[purposelycryptic]

Hi, and sorry for jumping in in the middle - Some others and I had been working until just recently on getting the anime-lists updated for the Summer 2019 season, as well as series from the Winter and Spring 2019 season and more, together with some other older series.

I know there were Pull Requests by @sven-7, and issues by sven-7 and @EndOfLine369 that had been addressed in various branches for some time, but until just a short while ago, we had no one with write-access consistently available to merge any Pull Requests since May; luckily, @slohj then came around and was able to perform the merges <( )> <( )> <( )> Currently, sixtenbe seems to have taken over.

I had actually considered just abandoning Scud_Lee's original project, and moving work over to my branch, between the months of total lack of Collaborator responses, and Scud_Lee missing in action since December, but as things seem to be moving again for the moment, and my own health continues to steadily deteriorate, that was put on hold.

Anyways, apologies for the overly-long, only semi-related introduction.

TL;DR: I'm not sure of this will help with TVDB4 mapping, as I'm ashamed to say I haven't kept up with ASS/HAMA development lately, and only use the basic AniDB mode on my server, but my next goal is to fix the mappings for the One Piece franchise - the core series is currently set to 'Absolute Episode', but, as you've experienced, AniDB and TheTVDB don't properly align 100%, and, since someone at TheTVDB thought it was a good idea to purge the values of the 'Absolute Episode Number' fields for the episodes of all the long-running series I've checked, it isn't a simple fix on that end.

All the mods at TheTVDB I've talked to insist that they lack the access to make big batch edits to the DB like that, when I've talked to them about fixing this and similar issues, and that no one actually editing/maintaining it has that level of access, but, since that data was purged from thousands of episodes, either someone was incredibly patient and devoted to removing that data episode by episode, or the mods are either misinformed or lying.

Regardless, manually mapping the series, and updating any outdated/incorrect mappings for the rest of the franchise, should fix issues with the series for anything relying on the anime-lists repo. Hopefully that includes this. It might take a few days (plus getting the Pull Request merged), as I have series of medical appointments these next days.

Shameless Help-Request: Due to months of minimal activity on the anime-lists repo, a lot of unfixed errors have popped up, and, while I manually went through the AniDB Season Chart for Summer 2019, I haven't yet had the time to do so for Fall 2018 or Winter an Spring 2019, and I know there is still unmapped content from those periods - if you run into anything like that, it would definitely help a lot if you could post it on the issue tracker.

I can't speak for anyone else, but for me, no need to do anything complicated or post corrected mappings, though that is obviously very helpful too, but a name and/or AniDB ID of the series/OVA/movie in question is incredibly useful as is, and I'll try to get it cleared up ASAP. The hardest part with errors and missing mappings is usually finding them

[sven-7]

someone at TheTVDB thought it was a good idea to purge the values of the 'Absolute Episode Number' fields for the episodes of all the long-running series I've checked, it isn't a simple fix on that end.

@purposelycryptic, what kind of series had absolute numbering removed?

With the new code, One Piece is now working great. It puts 590 mid-season, as it aired.

[purposelycryptic]

@sven-7 Off the top of my head, Saint Seiya, Ranma ½, Naruto + Shippuden, Bleach... It may actually be TVDB as a whole. There's just an empty spot where the number used to be:

It doesn't seem to create any problems as long as AniDB episode order and TVDB episode order match up, as absolute episode number can then be generated by simply adding the total number of episodes in previous seasons to the episode number in question the respective season; but having a set absolute episode number was helpful when they went out of sync, as (at least ideally), the AEN would account for that.

Anyway, I'm glad to hear One Piece is back in order again on your end - the anime-list mappings for it had other issues as well (the specials mapping was pretty scrambled, but the fix for that is already merged), which got me started on it, and then I saw this thread, and figured I may as well go over the whole thing. I'm still pretty sick these days, but simple stuff like adding/checking/fixing mappings at least keeps my brain busy:-)

[sven-7]

Ah - that is actually a bug in the display page of the actual website. Take any URL and add /edit to the end of it and even if it's locked, you'll see the correct data. It's likely the absolute number is still there, but the display on TheTVDB website is just broken. It's a known bug since they rolled out v2. v3 should fix it.

See:

[purposelycryptic]

Well, that definitely explains why AES somehow still worked - I always thought the explanation I was given was a little odd...

Weird that some mods didn't even seem to know about that; looks like we fully mapped some stuff (like Saint Seiya) for no real reason then... D'oh.

Now I'm left wondering if it would be easier to convince the mods to correct the AE numbering for One Piece, or to just map it - probably the latter.

[sven-7]

They won't remap that episode for One Piece (it's been tried before). I'm not sure why they treated 590 differently than 492 and 542, but they've deemed it a special and that's how it goes. I think it's because it's also episode 99 of Toriko, if I'm not mistaken. Anyway - HAMA/ASS now handles it well in tvdb4 mode and inserts it mid-season as it aired.

[purposelycryptic]

I thought it was one of those two-part episodes where one aired as part of One Piece, and the other as part of Toriko, but I could be getting it mixed up, as I usually skip over cross-promotional episodes like that.

But yeah, although they've gotten far better (and certainly more polite) over the years, as older, meaner and more overly-self-important, I-am-the-God-King type mods retired or became less active, getting anime entries changed on TheTVDB is still a lot like pulling teeth. These days, it's just more like pulling wolf-teeth, instead of dragon-teeth, but neither one is particularly fun.

But it sounds like the TVDB4 mapping issue is fixed, no teeth-pulling off any kind required, which is always a plus :)

[architdate]

An update on the issue, I had to revert the code back to the latest commit and then plex dance again to clear out all data and then remodify it again using the lines sent by @ZeroQI . After that tvdb4 worked flawlessly. No duplicate images and folders were being recognized as they should based on the custom mapping specified. (even with sub folders) The only issue that I can see now is that for me the special name didnt load like it did for @sven-7 (ref ep 590 in the image below)

Apart from that it looks ready to be commited to master 👍 Thanks for all the help!

[ZeroQI]

@architdate so episode 590 titleis "Episode 590" for you... need hama custom series log in agent data folder to narrow down but excellent news for the rest

[sven-7]

The only weird thing I've found (and not a huge deal) is that TVDB4 specials (even not absolute) show up in the missing ep logs now.

[architdate]

@ZeroQI sorry for the delay in response, was on a trip so didn't check GitHub. I return tonight, so will send you the relevant logs

[sven-7]

@ZeroQI -- I think this can be committed when ready. It's been working well for me.

[ZeroQI]

tried to adjust the missing eps logs accordingly. updated master code. Please update if i messed it up

[sven-7]

Seems to still be placing them under missing episodes.

_Logs.zip