ZeroQI / Absolute-Series-Scanner

Seasons, absolute mode, Subfolders...
999 stars 155 forks source link

Putting specials between episodes #454

Closed Sawtaytoes closed 11 months ago

Sawtaytoes commented 11 months ago

Is your feature request related to a problem? Please describe. With "One Punch Man" and some other series, there are OVAs or specials which should be interleaved between episodes.

No matter which DB ID I put on the series, I can't seem to get them moved around properly. If I manually rename the episodes to fake their numbering, the episode metadata either doesn't exist or gets messed up.

Describe the solution you'd like I'd like a way to automatically interleave episodes using some public database, but if that doesn't exist, it'd be nice to pass in a file to say "these specials are actually episodes and come after episodes X".

Something like "Special 01 -> Episode 04" where it'd be placed after episode 4.

Describe alternatives you've considered I've thought about renaming episodes, so they're ordered properly, but all that does is mess up the metadata.

Additional context image

These 6 specials should appear between the Season 1 episodes. In doing so, I would only have one season, so clicking "One Punch Man" would give me all of Season 1's episodes with thumbnails rather than split into Specials and Episodes: image

ZeroQI commented 11 months ago

Specials between episodes is a Plex functionality not yet implemented. To my knowledge, only supported by XBMC using tvdb agent...

Sawtaytoes commented 11 months ago

Do you mean Plex doesn't support it or that the scanner/agent doesn't support it?

I'm wondering if there are other ways to handle it like having the specials named as part of season 1 and then having some XML, INI, or Markdown file coordinate them with the correct metadata.

ZeroQI commented 11 months ago

A scanner decides the season and ep number Plex would have to manage that and it doesn't support it...

Pick anidb numbering or tvdb numbering to start with... You want specials to magically insert itself and plex, scanner and agent do not support this

Sawtaytoes commented 11 months ago

From my understanding, Plex shouldn't have anything to do with this. It's probably that I don't understand how it works though.

I thought I could provide an XML metadata lookup file like "ScudLee mapping" where I can re-designate episode numbers to say "episode 4 is 5" and "special 1 is episode 4".

This is what ScudLee mapping looks like for One Punch Man season 2:

  <anime anidbid="12430" tvdbid="293088" defaulttvdbseason="2">
    <name>One Punch Man (2019)</name>
    <mapping-list>
      <mapping anidbseason="0" tvdbseason="0">;1-8;2-9;3-10;4-11;5-12;6-13;7-14</mapping>
    </mapping-list>
    <before>;1-1;2-3;</before>
  </anime>

It seems like there's no way to designate specials to remap with the way this XML file works. Is that what you're referring to?

ZeroQI commented 11 months ago

The XML mapping file map files between anidb and tvdb so the metadata matches. You do not get to chose what anidb or tvdb says , just mapping between them appropriately. Tdvb4 create default seasons (arks) for tvdb4 mode if seasons were not provided.

Sawtaytoes commented 11 months ago

If I'm understanding you, with tvdb4 I can create my own seasons and create my own seasons?

Does this mean I can combine my specials into season episodes or is it only for taking a single season and divvying it up into multiple?

ZeroQI commented 11 months ago

Yes. Will all episodes using absolute numbering (no season, just ep number) and tvdb4 forced ID, you dump them in season folders as you wish and it just works.

Specials will stay in the season 0/specials numbering

Tvdb5 would mash it up in chronological order but was only use full for a stars wars Anime that was misnumbered but could rename specials as well, to be tested

Sawtaytoes commented 11 months ago

Specials will stay in the season 0/specials numbering

What does this mean?


I used a bunch of text files and renamed them to .mkv for experimentation. I got the tvdb4 series ID. Then I created a Season 01 folder and threw the files in there.

How do I remap episode numbers such that I get metadata for specials?

Like this?

tvdb4.mapping:

01|001|012|Season 1

How do I tell it which OVAs from my Blu-ray go in between which episodes? Are you saying to renumber them all 1-18 instead of 1-6 and 1-12?

image


This is the correct ordering:

  1. One Punch Man - s01e01 - The Strongest Man.mkv
  2. One Punch Man - s01e02 - The Lone Cyborg.mkv
  3. One Punch Man - s00e02 - The Pupil Who Is An Extremely Poor Talker.mkv
  4. One Punch Man - s00e01 - A Shadow That Snuck Up Too Close.mkv
  5. One Punch Man - s01e03 - The Obsessive Scientist.mkv
  6. One Punch Man - s01e04 - The Modern Ninja.mkv
  7. One Punch Man - s01e05 - The Ultimate Master.mkv
  8. One Punch Man - s00e03 - The Ninja Who Is Too Complicated.mkv
  9. One Punch Man - s01e06 - The Terrifying City.mkv
  10. One Punch Man - s01e07 - The Ultimate Disciple.mkv
  11. One Punch Man - s00e04 - Bang, Who Is Too Overbearing.mkv
  12. One Punch Man - s01e08 - The Deep Sea King.mkv
  13. One Punch Man - s01e09 - Unyielding Justice.mkv
  14. One Punch Man - s00e05 - The Sisters Who Have Too Many Things Happening.mkv
  15. One Punch Man - s01e10 - Unparalleled Peril.mkv
  16. One Punch Man - s01e11 - The Dominator of the Universe.mkv
  17. One Punch Man - s01e12 - The Strongest Hero.mkv
  18. One Punch Man - s00e06 - The Murder Case That Is Too Impossible.mkv

Then "Road to Hero" comes after, which is a separate 45 minute movie that goes between season 1 and 2: One Punch Man - s00e07 - Road to Hero.mkv

Season 2 is similar to Season 1 with mixed episodes and OVAs.

ZeroQI commented 11 months ago

Thanks for the precise breakdown

If you follow tvdb numbering for specials they should get proper metadata.

There is to my knowledge no way to insert specials in between episodes apart from XBMC using tvdb agent as tvdb have the information, however there is no metadata for plex to have this information for third party agents in the metadata structure

After some research, DVD ordering seem to do that, using new plex tvdb agent libraries seemingly, which does not support legacy agents like hama https://www.reddit.com/r/PleX/comments/toy2pd/aired_order_vs_dvd_order_in_new_plex_tv_series/

Sawtaytoes commented 11 months ago

TVDB doesn't insert these OVAs (yes, only on disc, I think) between the episode listing even with DVD Ordering.

The reason I went with HamaTV and Absolute Series Scanner was actually to fix the improper episode ordering of One Punch Man :P. I have another show with an OVA at episode 13 in-between others. It's called Episode 13.5 I think.

Is ASS able to handle ".5" episodes? If so, maybe I can somehow make these OVAs as 0.5 and setup a mapping file to tie them back to the special.

I'm wondering if there's any other way to set the metadata of episodes including specials.

ZeroQI commented 11 months ago

ASS need to give season and episode numbers as supported by Plex . 5 is not supported

I believe you take the problem backwards.... Pick anidb or tvdb and number as per the website... That is the expected way. Now I made some forced modes to add flexibility but I am sorry I cannot see you getting what you want here...

For this, you should go in Plex forums and ask people given your example before how they number things...

Sawtaytoes commented 11 months ago

I've been working on this for a couple weeks and have a list of what I want. I think aniDB is the correct DB for me. It's all-inclusive and always has lookups for everything I want.

Question: Can you add IDs to episodes? I know you can add them to the series, but what about [anidb-e143322].mkv?

I'm thinking, if there's some file that lets me map aniDB values to my own Series (not seasons), I can figure this out.

Summary

  1. Take two aniDB Series and combine them into a custom Series. That way, 2 cours for Season 2 show up as one Season 2 entry.
  2. Take episodes from aniDB IDs or s00e01 and create my own single-season Series that orders OVAs and episodes together.
  3. Split Series into multiple Series such as 1-13 and 14-52 for Armored Troopers VOTOMS and inject an OVA movie (The Last Red Shoulder) in-between those episodes as part of the Collection.

A new way to group

To do this, I don't need to change the anidb IDs on my folders. All we're doing is virtually grouping episodes based on some INI or XML file.

HamaTV or something is creating collections off the ASS data. If we leave the ASS data and folder structure untouched while adding an XML file, we should be able to achieve the 3 methods I show below.

1. Create New Series

Create a new "Re:Zero Season 2" single-season Series that looks like this:

Re:Zero (2020) s01e01
...
Re:Zero (2020) s01e12
Re:Zero (2021) s02e12
...
Re:Zero (2021) s01e13

2. Interleave OVAs

The series "One Punch Man (2019)" is a single-season Series that looks like this (note the season numbers):

One Punch Man (2019) s00e01
One Punch Man (2019) s00e02
One Punch Man (2019) s01e01
One Punch Man (2019) s01e02
One Punch Man (2019) s00e03
One Punch Man (2019) s00e04
One Punch Man (2019) s01e03

3. Split Series to create 2+ new ones

Armored Troopers VOTOMS should have "The Last Red Shoulder" OVA occur between episodes 13 and 14, but it's a separate movie OVA, not part of Season 1. That means we need to utilize Collections:


Armored Troopers VOTOMS Collection
  | Armored Troopers VOTOMS [eps1-13]
  | Armored Troopers VOTOMS: The Last Red Shoulder
  | Armored Troopers VOTOMS [eps14-52]```

With ASS and HamaTV, we can auto-name split series by their episodes in Plex as shown above.

Like this:
<img width="599" alt="image" src="https://github.com/ZeroQI/Absolute-Series-Scanner/assets/3948069/b0ff0c68-5695-4676-9641-516420f9ea4d">
ZeroQI commented 11 months ago

Scanner chose series title, season, and episode numbers. Agent search function choose unique ID (tvdb-xxxxx or anidb-xxxx for Hama) Agent update function give a list of metadata per season and episode number according to the unique id

This is the reality. You are meant to pick a metadata source and modify the numbering to conform to it. You are trying to make plex to support Specials between episodes. XBMC does that not Plex.

It is not possible for plex to support forced Ids for episodes, although my YouTube agent does something approaching, this is not really adaptable to Hama... Anidb3/4 could create something like separate seasons for each anidb entries but that is dodgy...

You should ask on the forum how they do for some particular series, somebody might have managed to do something usable...

You can also specify an alternate agent for one series if one works better than Hama...

Sawtaytoes commented 11 months ago

I asked in Plex support. No one seems to have this solved. Some people say "it works correctly". When one person told me how "works correctly" is setup, it's exactly what's in TVDB (which is wrong).

It lets specials play between episodes, but only if TVDB has it setup, and only if you click "Play" on the series. If you click and episode manually, it won't suddenly start playing the OVA instead, and there's no visual indication for how it's supposed to look.


On the comment about anidb3/4, I'm assuming you mean a hack to makea bunch of 1-episode seasons in a collection? I mean, that would let me put movies in between, but like you said, super hacky.

Although... Maybe not so bad? It would be super strange, but if each episode was treated like a movie, then everything would be on equal footing. Other than the fact that posters would be completely messed up, I could use HamaMovies instead of HamaTV and slot in every episode and OVA in the right spot.

Not only that, it could also fix the situation with multiple versions of the same episode.

Neon Genesis Evangelion has 2 versions of episodes 21-26. I was thinking of making a fake "Season 1" that had 1-20 and a fake Season 2 with the original 21-26 and a fake Season 3 with the Director's Cut 21-26. But that would be pretty hard because it's two episode 21s, and I'd have to split between Aired and DVD ordering in TVDB for it to get the correct metadata.

But if I had them separate "movies", then I could set up the ordering in the collection however I want. Auto-play works with Movie Collections too!


If Plex allowed collections of collections, we'd be in a great spot because I could separate group movies into "Seasons" and order the episodes however I wanted. The only way to do that today is with "Series" and "Seasons", but then it gets hacky like I was saying.

I think trying to make this work with episodes is probably the safest bet because making episodes first-class citizens with movies could have unforeseen consequences.

Unless I'm wrong?

Sawtaytoes commented 11 months ago

I thought about something else.

Going through Hama's code for TVDB4 mapping, it seems as though I misunderstood this functionality. All this does is take a series of many episodes and defines the seasons and titles for those seasons.

It's not reordering episodes, it's only organizing the seasons differently. Does it renumber the episodes for each season starting back at 1? If so, how's that accomplished?

Is it possible to reorder episodes with Hama? Can I say "episode 6 on disk is going to appear as episode 7 in Plex"? Because if you can, then it should be possible to make a special into an episode by giving it an episode number (making it not a special anymore).

ZeroQI commented 11 months ago

If tvdb numbering is not for you, then anidb in collections is the way to go, with collections from the movie collection xml improved with Anidb relationships, and completed by hand as it will be far from perfect. This will not show the order perfect but perfect viewing order is not possible Tvdb4 is to have a super long running series using absolute numbering so no season split into arcs so you do not need to scroll hundreds of eps on a tablet pc or phone... One piece is a great exemple with clearly defined arcs... The tvdb4.xml have good examples of the default seasons used without users having to create season folders themselves... The episode number doesn't change, ASS just move the episode in the correct season as the scanner responsability is the series name, season and episode number for eps... Literally the first 2 lines in the readme

Season title is coded but not updated by Plex, if it changed that would be great...

Hama do not reorder stuff. It is an agent that provide metadata to each episode so it need to match anidb or tvdb.

You have to follow anidb or tvdb numbering You have to start from the metadata sources and what plex allows, you cannot start from what you want from a logical standpoint....

Sawtaytoes commented 11 months ago

Related to what you were saying, Plex exposed titled Seasons:

image

Sawtaytoes commented 11 months ago

I think I understand then.

ASS sets up the season and episodes and the ID (aniDB or TVDB). Hama takes those IDs and populates metadata based on what you have available.

So if I want to modify episode numbers and configure my own database, I'd need an intermediary API that can do a translation between aniDB episode numbers, and my preferred mapping right?

I'm wondering if I can fake this somehow. I can't add IDs to episodes in Plex, but I could create a file that lists each on-disk episode number and the aniDB episode number it maps to. Then HamaTV only needs to populate metadata based on that file rather than the aniDB series ID right?

Example episodes.mapping file:

01|anidb-e174024 # episode 1
02|anidb-e171297 # episode 2
03|anidb-e174024 # special 1

This would let me put any episode or OVA or movie in place of an episode with the correct metadata in whatever order I want so long as Hama can read this episodes.mapping file. When it looks up metadata, all it needs is the series ID and each episode ID.

ZeroQI commented 11 months ago

The agent in two parts assign the ID then use it to download metadata Only when it is a date based numbering can the metadata agent set the date and modify the season number/episode but buggy if multiple eps have the same published date

If you want to modify the ep number in Plex the scanner need to do that

It would shift everything so the agent would need to look it up per file... I have no plan to include this monstrosity in ASS or HAMA...

Sawtaytoes commented 11 months ago

Right now, the lookup happens where it pulls in the series number. It gets all episode data in 1 call.

In what I'm describing, you have to make a separate call for each episode based on a separate ID?

Hmm...

What about this:

episodes.mapping:

01-02|01-02
03|anidb-e174024 # special 1
04-05|03-04
06|anidb-e... # special 2

Now you can get all episode data and only have to look up a few specials.

Not sure how easy this is to do in Python. I'm a JavaScript developer, and setting something up like this with RxJS would take minutes.

Another option

What about mapping IDs to the data call?

I'm assuming you're making 1 data call for the whole series that includes episodes like this:

image What if you were able to set the episode number by creating an array or dictionary of the episode ID in the data call to the new number?

I can try to make my own plugin too if this seems like too much.

My goal right now is to understand what's so hard about this and what I'm missing.

ZeroQI commented 11 months ago

Read ASS code and then HAMA code. It is awfully complex due to anidb and tvdb mapping and cross referencing them both with anime-lists xmls so regardless of the forced ID both metadata sources are used...

I kept it functional while adding forced tvdb modes so people don't have to rename eps for absolutely numbered series

I can't get most users to rename eps Plex itself doesn't support the functionality but you want to code a database per episode so the ordering is correct in Plex display? I consider this not feasible within HAMA and not worth it time wise as users will rather look up the viewing order than rename each episode file to include a forced ID and update a database... While it would be useful for some series, users would put series in collection and look up the viewing order...

If you mess up the ep number the agent has to know you did that and the metadata source won't match metadata to the right episode number....

Sawtaytoes commented 11 months ago

Yeah, it's definitely complex. I've spent some time the last 3 nights trying to understand it, and there's a lot going on in many files. If it was simpler, I probably wouldn't have bothered you and would've tried it myself :p.

I also don't know Python, so that's another barrier to entry.

Overall, many use cases are solved by simply using aniDB rather than TVDB since "seasons" aren't the same thing in Japan as other places. Also, movies are often part of the watch order between seasons, but even between episodes. It's the "between episodes" part that I'm really trying to solve somehow.


You're talking about renaming episodes on disk and creating a database to map them, but what about the other way around? What if my names were aniDB or TVDB on disk, but the episode numbers were changed when reported to Plex? Wouldn't that be easier? But somehow, Hama needs to know that episode 10 is actually episode 9 in the metadata right?

It's a similar change, but I don't have to rename any files, only control an INI or XML file. Like you said, it could be a lot of work for little return.

ZeroQI commented 11 months ago

Plex is the only tool with accessible 3rd party scanner and I may have the most experience. If you forego the recursive calls to support grouping folders the scanner is simple but you will need another agent. You will need to change the ep number and it will not match any metadata source like anidb or tvdb. It is complex and not worth it for me... Little return when you can just look up watching order using anidb entries in collection in Plex..

Sawtaytoes commented 11 months ago

Can I donate $300 to get that functionality added? If it's something you're not fond of doing and something specific to my needs, I don't mind paying for your time.

It would help me finally take movies and specials and put them between episodes in a season.

Watch order is more important to me than how Plex numbers episodes. I want to click "Play" and not have to look up a watch order. Ideally, episodes would be numbered correctly, but that's simply not possible in Plex, so if we have to renumber episodes, I'm fine so long as they play in the correct order.

Goals

Using an ID system for some specials and movies (OVAs), I can place them where I want in a season.

  1. Having Specials show up between episodes.

Example: One Punch Man. I'm fine if episode 12 says episode 17 in Plex because there are 5 specials that come before it. This way preserves watch order.

  1. Having aniDB OVAs or movies, which aren't specials, able to be placed between episodes.

Example: Re:Zero's Memory Snow OVA is episode 1 of Re:Zero (2018) [anidb-13430] yet it's supposed to go between episodes 6 and 7 of Re:Zero [anidb-11370]. It is okay if Plex shows Memory Snow as episode 7 and episode 7 is now episode 8 because the watch order is preserved.

Example 2: "The Frozen Bond" (episode 2 of Re:Zero (2018) [anidb-13430]) is supposed to occur after Re:Zero [anidb-11370] finishes. It's a feature-film and not part of Re:Zero [anidb-11370], I would like it to be a single-season single-episode in Re:Zero (2018) [anidb-13430] just like other movies. What I mean is, I don't want Memory Snow (episode 1) to show up in this season because I've mapped it to Re:Zero [anidb-11370]. It's fine if this is a separate configuration in the Re:Zero (2018) [anidb-13430] folder.

I think that's everything off the top of my head.

The rest is already possible thanks to ASS and Hama using aniDB IDs and Plex Collections.

Sawtaytoes commented 11 months ago

I came up with another idea that might be the easiest way.

What if I wrote my own API that returns data just like aniDB? All I need to do is tell Absolute Series Scanner the URL to my own API instead.

That way, given an aniDB ID, I can return whatever episodes I want in whatever order I want that includes the correct metadata. This is different from an XML file because it's my API that maintains the watch order and episode listing, so it should be able to pull the correct metadata that way as well.

Do you think that'd work? Other than me changing the URL in my local version of the plugin, it shouldn't require any changes on your end. Just so long as I understand exactly which APIs to change. Where are those files located that make the API calls?

ZeroQI commented 11 months ago

I wrote lambda that has all metadata in XBMC. Nfo format, but nothing exists that remap manually episodes and seasons...

People in your situation would duplicate the files in another folder to rename it properly.

After all, if you check any other ecosystem with agents, you have anidb or tvdb only agent and season and episode must match anidb or tvdb...

You could write a simple scanner to remap filenames to a given season and episode number but metadata won't match. If you use something like XBMC.nfo for metadata an agent could read the metadata that doesn't match anidb nor tvdb for the ep number...

Sawtaytoes commented 10 months ago

Oh! So as long as I wrote my own script to generate and update an .nfo file, I could use that in Plex for metadata? That's great! Then I don't have to worry about aniDB and TVDB, I can use my own JavaScript code to generate them!

Or are you saying that's only available in XBMC; Absolute Series Scanner and Hama don't look at a metadata.nfo file?

ZeroQI commented 10 months ago

I am saying you can write your own scanner and agents and Lambda does it with XBMC.NFO files but you can code anything. If you did a clever format you could do a loop like in Hama where it loads all the medatada dicts from the different sources and process the metadata...

Sawtaytoes commented 10 months ago

Do you have an example of one of these XMBC.NFO files?

I'll write something to get aniDB metadata in there! Since I work in the JavaScript world, async data calls and API connections are perfect for me! Can't wait to fix my watch order!

ZeroQI commented 10 months ago

https://kodi.wiki/view/NFO_files/Templates

Sawtaytoes commented 10 months ago

Just to confirm, does this work with ASS and Hama or should I write my own? Or do you know of any that already look at .nfo files?

ZeroQI commented 10 months ago

I google "Plex xbmc. Nfo" https://github.com/gboudreau/XBMCnfoMoviesImporter.bundle

Lambda also reads and write xbmc.nfo files https://github.com/ZeroQI/Lambda.bundle

Sawtaytoes commented 10 months ago

I'd rather use your plugin. I didn't realize what you meant, but now I see that Lambda is another of your plugins 👍.

ZeroQI commented 10 months ago

No functionality to remap episode or seasons within the scanner or agent, mapping file match anidb to tvdb only. The nfo functionality is in Lambda already so you could do a scanner code that maps filenames to season and ep number