ZeroQI / Absolute-Series-Scanner

Seasons, absolute mode, Subfolders...
1.02k stars 155 forks source link

Plex scanner responsibilities

A Plex Series Scanner makes the video files showing in Plex and populate the following for the video files:

If a file is not showing in plex or showing at the wrong season and/or episode number, or is not passing through the forced id, then it is a scanner issue. Anything else is metadata related and agent specific.

If you post anything on the scanner github or thread for any issue like poster missing/wrong, episode thumbnail/screenshot/title/summary missing/wrong, then you clearly haven't read this and need to pay the RTFM tax by donating (or you just find this the best anime scanner and/or agent, much apreciated average is 5 euros). Also applies if you want assistance but didn't follow troubleshooting steps or include logs...

Plex Agent responsibilities

A Plex metadata agent will:

Which Metadata/Title source to select?

Absolute series scanner functions that differes from Plex Series Scanner

Requirements

File Naming Conventions / Numbering

This scanner supports absolute and season numbering, but here are two references for guidelines

Files

File naming convention Template / Folder Example
Splitting folders: 0-9 0-9, A,...,Z folder. Add EACH as folder. Do not use the parent folder
Episode Name Pattern: Season %S/%SN s%0Se%0E Season 2/Show Name s02e03.ext
Multi-Episode style: Extend Season 2/Show Name s02e03-05.ext
Multi-part episodes: cdX, discX, diskX, dvdX, partX, ptX Season 2/Show Name s02e03 - pt1.ext
Multi-Media Version: Movie Name (year) - 1080p.ext Movie Name (year) - 1080p.ext
Specials scrapped: Specials, Season 0 s00e01/OP1/Ed3a/NCOP/S01/S1.ext
Other non scrapped: Extras Extras/Show Name xxxx.ext
BD rips /path/to/series-library/Series Name Season 2 Series.Name.Disc1.S02.E01-E12/BDMV/STREAM
Specials

Hama is a anidb (single season) & tvdb (multiple seasons) agent so either naming convention is fine. It will detect either successfully but you can convert one convention to the other whicle displaying by forcing ids (further down)

Let's use One piece special "Heart of Gold":

TVDB seasons

Anidb (single) season

####### Anidb type special numbering is detailed below:

Type Internal letter Episode number
Specials S Episodes 001-100
OPs C Episodes 101-150
EDs C Episodes 151-200
Trailers T Episodes 201-300
OPs/EDs P Episodes 301-400
Others O Episodes 401-500
unmapped Episodes 501-600
One libray for both movies and TV series

Movie files in Series libraries (since this is a Series Scanner) are supported if:

Season folders

Local media assets

It is supported but through "local media assets" agent, add it and and put it before HAMA in the priority order.
https://support.plex.tv/hc/en-us/articles/200220717-Local-Media-Assets-TV-Shows

Data type Source Comment
fanart art/backdrop/background/fanart-1.ext -1 can be ommited (same level as Video TS)
Series poster Series folder: Show name-1/folder/poster/show.ext (jpg, jpeg, png, tbn)
Season poster Season folder: Season01a.ext (jpg, jpeg, png, tbn)
Banner banner/banner-1.jpg
Theme song theme.mp3
Subtitles file name.ext (srt, smi, ssa, ass)
Plexignore files .plexignore

Movie libraries can have "Extra" in a specifically named folders or with the following at the end of the filename (hyphen important, no space afterwards):

Note: "Extras" folder is skipped by the absolute series scanner, put unsorted files in there, it won't show up in Plex

Grouping folder

Forcing the movie/series ID

Hama supports the following guid_type:

You can specify the guid to use the following way:

Note: You may use either the old-style Absolute-Series-Scanner format with square brackets ([anidb-###]) or the new style Plex-Folder-Id format with curly braces. ({anidb-###}) They are both interchangable to help people with converting to the new format.

guid_type Real file numbering Seasons numbering Episodes numbering Use case (example)
anidb Absolute 1 AniDb Force the anidb series id. Series will follow anidb episode numbering convention including specials
  • Sword Art online {anidb-8692}
anidb2 Absolute TVDB TVDB Map Anidb entries to the tvdb at the right season and ep number to show one Plex series entry only (need to be within a single tvdb entry)
  • Sword Art online {anidb2-8692}
  • Sword Art online II {anidb2-10376}
  • Sword Art Online Extra Edition {anidb2-10022}
  • Gekijouban Sword Art Online: Ordinal Scale {anidb2-11681}
anidb3 Absolute TVDB TVDB Uses ScudLee mapping to map the AniDB series to TVDB entries BUT overrides the mapping for TVDB season 0 entries and puts them in AniDB relational order by appending to existing seasons or adding new seasons at after the last TVDB season
  • Date a Live {anidb3-8808} => TVDB s1
  • Date a Live: Date to Date {anidb3-9734} => TVDB s0e1 -> s1e101
  • Date a Live II {anidb3-9935} => TVDB s2
  • Date a Live II: Kurumi Star Festival {anidb3-10568} => TVDB s0e2 -> s2e101
  • TBD (prep entry in TVDB) => TVDB s3
anidb4 Absolute TVDB TVDB Uses ScudLee mapping to map the AniDB series to TVDB entries BUT overrides the mapping for TVDB seasons entries and puts them in AniDB relational order by inserting new seasons and pushing later TVDB seasons back
  • Date a Live {anidb3-8808} => TVDB s1 -> s1
  • Date a Live: Date to Date {anidb3-9734} => TVDB s0e1 -> s2
  • Date a Live II {anidb3-9935} => TVDB s2 -> s3
  • Date a Live II: Kurumi Star Festival {anidb3-10568} => TVDB s0e2 -> s4
  • TBD (prep entry in TVDB) => TVDB s3 -> s5
tvdb Season TVDB TVDB Force the tvdbid, series will follow tvdb episode numbering convention including specials
  • Sword Art Online {tvdb-259640}
  • Season 1 - Sword Art Online [1-25]/ep ##.ext with ## from 1 to 25
  • Season 2 - Alfheim & Gun Gale Online [1-25]/ep ##.ext ## from 1 to 25
tvdb2 Absolute TVDB TVDB for absolute numbering displayed virtually as tvdb numbering, episode number resets to 1 each season, for series like Sword art Online(1-50, will be split into Season 1 [1-25] and Season 2 [1-25])
  • Sword Art Online {tvdb2-259640}/Ep ##.ext with ## from 1 to 50
tvdb3 Absolute TVDB Absolute For absolute numbering episodes displayed virtually using tvdb season numbering for long running series like One piece (1-700+), will be split into seasons while keeping the absolute episode number intact without having to create seasons in the real folder
  • Metantei Conan {tvdb3-72454}
tvdb4 Absolute, random season Abs/Custom/Arc db Absolute For absolute numbering episodes displayed using series arc as season for long running series with arcs like Dragon Ball Kai, or separated anidb series considered as half seasons by thetvdb (like 'Seraph of the end' numbered 1-24 splitted into 2 seasons). Will take the arc definitions from tvdb4.mapping.xml and posters from tvdb4.posters.xml unless the absolute numbered episodes were placed in season folders already
  • One Piece {tvdb4-81797}
tvdb5 TVDB Absolute Absolute TheTVDB Absolute numbering order (most useful for Star Wars: The Clone Wars, First ep is s02e15...) will remove seasons present and use the 'absolute_number' tvdb field order to re-sort the episodes. Allow to insert specials in between episodes too (prequel)
  • Star Wars: The Clone Wars {tvdb5-83268}
youtube YouTube None None Put Playlist id (PL... 2+16/32 chars long) on series folder or season folder (auto-reversing) or channel id on series folder (year used as season, added as date-based unless there are duplicates for the date in which case it choose ep number MMDDxx with XX being incremental)
youtube2 YouTube None None Recommended for channels that release multiple episodes per day. Put channel id on series folder (year used as season, episodes added as MMDDhhmm. If date not present in filename, will use the files date for Month, Day, Hour and Minute, but if the date is present in the filename than Month and Day are pulled from it instead)

You can force the tvdb4 mapping with a "tvdb4.mapping" file in the series folder containing the mapping format. Because it doesn't have an XML extension, it won't accept XML tags inside. Format exemple is fom anime tvdbid="331753" name="Black Clover":


    01|001|013|Introduction Arc
    02|014|019|Dungeon Exploration Arc
    03|020|027|Royal Capital Arc
    04|028|039|Eye of the Midnight Sun Arc
    05|040|050|Seabed Temple Arc
    06|051|065|Witches' Forest Arc
    07|066|072|Hot Springs Training Camp Arc
    08|073|084|Royal Knights Arc
    09|085|120|Reincarnation Arc
    10|121|121|Arc 10 (unknown length)
Filebot Xattr series id support

Filebot support metadata in file system extended attributes, and has released a movies, series scanner and secondary agent here: https://github.com/filebot/plex-agents/blob/master/Scanners/Series/FileBot%20Xattr%20Series%20Scanner.py

One can instruct to save metadata in the OS file system metadata (examples attached)

Advanced modes

If the season and/or episode numbers of your files don't match up with the AniDB or TVDB episodes, you can manually specify the season and the episode offset in the parent folder name.

The following suffixes are supported:

Examples:
== Example 1 ==
- "Bakuman      {anidb2-7251}" = "Bakuman      {tvdb-193811-s1}" = "Bakuman {tvdb-193811}"
- "Bakuman 2011 {anidb2-8150}" = "Bakuman 2011 {tvdb-193811-s2}"
- "Bakuman 2012 {anidb2-8836}" = "Bakuman 2012 {tvdb-193811-s3}"

== Example 2 ==
- "Sailor Moon Crystal {tvdb2-275039}"
- "Sailor Moon Crystal Season 3 {anidb2-11665}" = "{tvdb-275039-s3}" | "{tvdb2-275039-s3}" (depending if you keep absolute numbered eps in seasons)

== Example 3 ==
- "Bleach                                    {tvdb3-74796}"
- "Bleach movie 1 Memories in the Rain       {tvdb3-74796-s0}"
- "Bleach movie 2 The Diamond Dust Rebellion {tvdb3-74796-s0e4}"

== Example 4 == tvdb4 Custom selected Arcs as seasons (as tvdb use them as half seasons for black lagoon for example)
  The arc definition to split into seasons the absolute numbering is done using the following order:
    - Seasons folders manually created by the user with absolute numbered episodes inside (seasons already mapped manually)
    - in a local "tvdb4.mapping" file inside series folder with the following format lines, one per arc/season:
      \|\|\|\(optional)
    - without doing anything using the online arc database [github tvdb4.mapping.xml](https://github.com/ZeroQI/Absolute-Series-Scanner/blob/master/tvdb4.mapping.xml)

    <anime tvdbid="79604" name="Black Lagoon">
      01|001|012|The First Barrage
      02|013|024|The Second Barrage
      03|025|029|Roberta's Blood Trail
    </anime>
    <anime tvdbid="289906" name="Seraph of the End">
      01|001|012|Vampire Reign
      02|013|024|Battle in Nagoya
    </anime>

  "Black Lagoon                         {tvdb4-79604}"
  "Black Lagoon - The Second Barrage    {tvdb4-79604-s2}"
  "Black Lagoon - Roberta`s Blood Trail {tvdb4-79604-s3}"

  "Seraph of the End - Vampire Reign    {tvdb4-79604}"
  "Seraph of the End - Battle in Nagoya {tvdb4-79604-s2}"

== Example 5 ==
  "Hand_Tool_Rescue {youtube2-UCasG9kJWi1eVxM0QkyqKVJQ}"

.plexignore files

https://support.plex.tv/articles/201381883-special-keyword-file-folder-exclusion/

Plex system folder location

Source: https://support.plex.tv/articles/202915258-where-is-the-plex-media-server-data-directory-located/ Location:

You can actually move the plex system folder/storage/database here: (i give no warranty it works, you lunatic!)

Install / Update

Note:

Linux install script example


mkdir -p '/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Scanners/Series'
wget -O '/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Scanners/Series/Absolute Series Scanner.py' https://raw.githubusercontent.com/ZeroQI/Absolute-Series-Scanner/master/Scanners/Series/Absolute%20Series%20Scanner.py
chown -R plex:plex '/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Scanners'
chmod -R 775 '/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Scanners'

Troubleshooting:

Logs

Absolute series Scanner saves its custom logs in this relative path: Plex Media Server\Logs\ASS Scanner Logs...

If the scanner crash, you will get either no files (library creation) or no change (library already created) and will need to attach:

To have logs saved per library and also a log is not created per series, you might need to create a Plex token id file (Never share your Plex token)

For issues series specific (indicate if file not present, but create a token id file, see above):

You will find per-serie logs there with the following extensions as HAMA and ASS gather all logs in one place

And post in either:

Known issues:


Nov 16, 2016 18:48:53.594 [0x7f48c2324800] DEBUG - Adding subdirectory for scanner: /home/plex/things/anime/Ah! My Goddess 2
Nov 16, 2016 18:48:53.597 [0x7f48c2324800] ERROR - No module in VideoFiles
Nov 16, 2016 18:48:53.597 [0x7f48c2324800] ERROR - Error scanning directory .
Nov 16, 2016 18:48:53.597 [0x7f48c2324800] ERROR - No module in Absolute Series Scanner
Nov 16, 2016 18:48:53.598 [0x7f48c2324800] ERROR - We got an error scanning in /home/plex/things/anime

You bloody downloaded the web page and not the actual py file:


Jul 23, 2016 12:55:54.558 [5288] ERROR - Error scanning directory .
Jul 23, 2016 12:55:54.574 [5288] ERROR - No module in Absolute Series Scanner
Jul 23, 2016 12:55:54.574 [5288] ERROR - Error in Python: Looking up module:
Traceback (most recent call last):
File "C:\Users\Administrator\AppData\Local\Plex Media Server\Scanners\Series\Absolute Series Scanner.py", line 8, in 
from lxml import etree # fromstring
ImportError: DLL load failed: The specified module could not be found.

On windows install https://www.microsoft.com/en-us/download/details.aspx?id=48145 or other runtimes


2016-06-29 23:30:09,104 (30c) : CRITICAL (core:574) - Exception while loading code (most recent call last):
File "C:\Program Files (x86)\Plex\Plex Media Server\Resources\Plug-ins-a17e99e\Framework.bundle\Contents\Resources\Versions\2\Python\Framework\core.py", line 563, in load_code
self.init_code = self.loader.load(self.init_path, elevated, use_xpython = Framework.constants.flags.use_xpython in self.sandbox.flags)
File "C:\Program Files (x86)\Plex\Plex Media Server\Resources\Plug-ins-a17e99e\Framework.bundle\Contents\Resources\Versions\2\Python\Framework\code\loader.py", line 47, in load
code = self.compile(str(source), str(uni(filename)), elevated)
File "C:\Program Files (x86)\Plex\Plex Media Server\Resources\Plug-ins-a17e99e\Framework.bundle\Contents\Resources\Versions\2\Python\Framework\code\loader.py", line 52, in compile
return RestrictedPython.compile_restricted(source, name, 'exec', elevated=elevated)
File "C:\Program Files (x86)\Plex\Plex Media Server\Resources\Plug-ins-a17e99e\Framework.bundle\Contents\Resources\Platforms\Shared\Libraries\RestrictedPython\RCompile.py", line 115, in compile_restricted
gen.compile()
File "C:\Program Files (x86)\Plex\Plex Media Server\Resources\Plug-ins-a17e99e\Framework.bundle\Contents\Resources\Platforms\Shared\Libraries\RestrictedPython\RCompile.py", line 68, in compile
tree = self._get_tree()
File "C:\Program Files (x86)\Plex\Plex Media Server\Resources\Plug-ins-a17e99e\Framework.bundle\Contents\Resources\Platforms\Shared\Libraries\RestrictedPython\RCompile.py", line 59, in _get_tree
tree = self.parse()
File "C:\Program Files (x86)\Plex\Plex Media Server\Resources\Plug-ins-a17e99e\Framework.bundle\Contents\Resources\Platforms\Shared\Libraries\RestrictedPython\RCompile.py", line 56, in parse
return niceParse(self.source, self.filename, self.mode)
File "C:\Program Files (x86)\Plex\Plex Media Server\Resources\Plug-ins-a17e99e\Framework.bundle\Contents\Resources\Platforms\Shared\Libraries\RestrictedPython\RCompile.py", line 38, in niceParse
compile(source, filename, mode)
TypeError: compile() expected string without null bytes

Rights issues on Windows 2008 R2, solved by changing rights for:

On linux (and Mac OS-X), permissions issues could prevent the scanner execution, but after creating folder and setting proper permissions, all was working. Exemples solved by creating folders and setting proper permissions:

Using the latest binhex plexpass docker returns: https://github.com/binhex/arch-plex


Error in Python: Looking up module:
Traceback (most recent call last):
File "/config/Plex Media Server/Scanners/Series/Absolute Series Scanner.py", line 20, in 
from lxml import etree # fromstring
ImportError: libexslt.so.0: cannot open shared object file: No such file or directory

Using the latest binhex plexstandard docker returns: https://github.com/binhex/arch-plex


Error in Python: Looking up module:
Traceback (most recent call last):
File "/config/Plex Media Server/Scanners/Series/Absolute Series Scanner.py", line 20, in 
from lxml import etree # fromstring
ImportError: libexslt.so.0: cannot open shared object file: No such file or directory

Solution: Opening the dockers terminal and running "pacman -S libxslt --noconfirm" then rebooting the docker fixes the issue.

Task list

Reference for editing Read-Me: Link to Markdown or https://help.github.com/articles/basic-writing-and-formatting-syntax/

Donation link: https://www.paypal.com/cgi-bin/webscr?cmd=_donations&business=S8CUKCX4CWBBG&lc=IE&item_name=ZeroQI&item_number=Absolute%20Series%20Scanner%20%2b%20Http%20AniDB%20Metadata%20Agent&currency_code=EUR&bn=PP%2dDonationsBF%3abtn_donateCC_LG%2egif%3aNonHosted