collections not being generated

ecto1a2003 commented 4 years ago

plex version : qnap v1.19.3

file structure: v:/internet/%(uploader)s/%(playlist_title)s [%(playlist_id)s]/%(playlist_index)s - %(title)s [%(id)s].%(ext)s

library root: v:/internet/ examples: \Peachsaliva\baba is you [PL6tgmi2pvbIdsvT1GJ5YwpJCayWTl08p6] \Peachsaliva\Undertale [PL6tgmi2pvbIfaFxX8_QNwgh7pOXybL8oX]

\PlagueOfGripes\Animations [PLPBmbs-JPhHc-__sPdp3seXz-MWOVJP07] com.plexapp.agents.youtube.log com.plexapp.system.log \PlagueOfGripes\Dragonball Gripes [PLPBmbs-JPhHfF-d08nm_NY7Q1kO7vqG4r]

donation to follow shortly

ZeroQI commented 4 years ago

Thank you very much for the donation.

It is funny, I previously got 15+5+5+20+5=50€ euros donations for the agent (from the first 2018-10-16)
You exactly doubled that :D

Thank you for:

putting logs with the exemple in it
describing properly the issue including an impacted example

The code hints at what is going on:

2020-05-10 13:05:30,729 (7fd7f1e10700) :  INFO (__init__:181) - =============================================================================================================================================================
2020-05-10 13:05:30,730 (7fd7f1e10700) :  INFO (__init__:182) - update() - guid: PL6tgmi2pvbIdsvT1GJ5YwpJCayWTl08p6, dir: baba is you [PL6tgmi2pvbIdsvT1GJ5YwpJCayWTl08p6], metadata.id: youtube|PL6tgmi2pvbIdsvT1GJ5YwpJCayWTl08p6|baba is you [PL6tgmi2pvbIdsvT1GJ5YwpJCayWTl08p6]
2020-05-10 13:05:30,730 (7fd7f1e10700) :  INFO (__init__:60) - /share/CACHEDEV1_DATA/Videos/Internet/Peachsaliva/baba is you [PL6tgmi2pvbIdsvT1GJ5YwpJCayWTl08p6]/1 - Ruby Weapon Hour - Baba Is You (Blind) [xEJ8YggKzYA].mp4
2020-05-10 13:05:30,731 (7fd7f1e10700) :  INFO (__init__:72) - [!] Library access denied
2020-05-10 13:05:30,732 (7fd7f1e10700) :  INFO (__init__:80) - [!] ASS root scanner file missing: "/share/CACHEDEV1_DATA/.qpkg/PlexMediaServer/Library/Plex Media Server/Plug-in Support/Data/com.plexapp.agents.hama/DataItems/_Logs/_root_.scanner.log"
2020-05-10 13:05:30,732 (7fd7f1e10700) :  INFO (__init__:226) - [ ] dir:        "/share/CACHEDEV1_DATA/Videos/Internet/Peachsaliva/baba is you [PL6tgmi2pvbIdsvT1GJ5YwpJCayWTl08p6]"
2020-05-10 13:05:30,733 (7fd7f1e10700) :  INFO (__init__:227) - [ ] library:    ""
2020-05-10 13:05:30,734 (7fd7f1e10700) :  INFO (__init__:228) - [ ] root:       "/share/CACHEDEV1_DATA/Videos/Internet/Peachsaliva/baba is you [PL6tgmi2pvbIdsvT1GJ5YwpJCayWTl08p6]"
2020-05-10 13:05:30,734 (7fd7f1e10700) :  INFO (__init__:229) - [ ] path:       ""
2020-05-10 13:05:30,737 (7fd7f1e10700) :  INFO (__init__:234) - [ ] series_root_folder: "/share/CACHEDEV1_DATA/Videos/Internet/Peachsaliva/baba is you [PL6tgmi2pvbIdsvT1GJ5YwpJCayWTl08p6]/"
2020-05-10 13:05:30,738 (7fd7f1e10700) :  INFO (__init__:235) - [ ] subfolder_count:    "0"
2020-05-10 13:05:30,738 (7fd7f1e10700) :  INFO (__init__:258) - Grouping folder not found or single folder, root: /share/CACHEDEV1_DATA/Videos/Internet/Peachsaliva/baba is you [PL6tgmi2pvbIdsvT1GJ5YwpJCayWTl08p6], path: , Grouping folder: , subdirs: 0, reverse_path: ['']

The library file cannot be open and the code can't deduce the root, hence relative path, hence if there is a grouping folder with the depth. Logs do include a warning:

 (7fd7fbb69700) :  INFO (__init__:516) - Library: /share/CACHEDEV1_DATA/.qpkg/PlexMediaServer/Library/Plex Media Server
2020-05-10 13:05:30,548 (7fd7fbb69700) :  INFO (__init__:527) - Place correct Plex token in X-Plex-Token.id file in logs folder or in PLEX_LIBRARY_URL variable to have a log per library - https://support.plex.tv/hc/en-us/articles/204059436-Finding-your-account-token-X-Plex-TokenHTTP Error 401: Unauthorized
2020-05-10 13:05:30,548

Some OS just allow access to the library xml file, other need the token id... The link gives an outdated method, because plex changed, at least on my Plex running Synology NAS.

If you create a "Plex Media Server/X-Plex-Token.id" containing the Plex token id by logging on plex.tv/desktop, then https://plex.tv/devices.xml, it will be MediaContainer/Device ... token="xxxxxxxxxxxxxx".

Please let me know if it resolves And thanks again for the doubling the previous 5 donations :)

ecto1a2003 commented 4 years ago

I put that in the root of the library and Whamo! looks good so-far.

as far as donations go its tottaly worth it to support the tools is use.

ecto1a2003 commented 4 years ago

hmm, its only added 2 playlists to each collection and left other playlists seperate. com.plexapp.agents.youtube.log com.plexapp.system.log

ecto1a2003 commented 4 years ago

possibly due to special charectes in the playlist foldername? I left only the playlist id and refreshed metadata and it was pulled into the collection.

ecto1a2003 commented 4 years ago

Or because the name of the playlist has a colon in it on youtube but the folder can't naturally so they don't match.

ZeroQI commented 4 years ago

"X-Plex-Token.id" seem well placed since the root folder in library is not empty and collections generated

"Peachsaliva"
"PlagueOfGripes"

it test for grouping folder line 253: if len(reverse_path)>1 and not season_folder_first and subfolder_count>1: ### grouping folders only ###

Not generated for: Grouping folder not found or single folder, root: /share/CACHEDEV1_DATA/Videos/Internet, path: PlagueOfGripes/Let's (not) Plague - Dark Souls 2 [PLPBmbs-JPhHeXnH-QpICXf_4xb3MeDlzq], Grouping folder: PlagueOfGripes, subdirs: 5, reverse_path: ['PlagueOfGripes']

Grouping folder not found or single folder, root: /share/CACHEDEV1_DATA/Videos/Internet, path: PlagueOfGripes/Let's Plague - Dark Souls [PLPBmbs-JPhHeG6O1CbPn8TqFAZmZvpGT8], Grouping folder: PlagueOfGripes, subdirs: 5, reverse_path: ['PlagueOfGripes']

Grouping folder not found or single folder, root: /share/CACHEDEV1_DATA/Videos/Internet, path: PlagueOfGripes/Let's Plague - Dark Souls (Run 2) [PLPBmbs-JPhHcvI9t8mbFWbzkMI0JCwv6O], Grouping folder: PlagueOfGripes, subdirs: 5, reverse_path: ['PlagueOfGripes']

It looks like the playlist folder is removed from reverse_path lines 238-251

      reverse_path, season_folder_first = list(reversed(path.split(os.sep))), False
      SEASON_RX = [ 'Specials',                                                                                                                                           # Specials (season 0)
                    '(Season|Series|Book|Saison|Livre|S)[ _\-]*(?P<season>[0-9]{1,2}).*',                                                                                 # Season ##, Series #Book ## Saison ##, Livre ##, S##, S ##
                    '(?P<show>.*?)[\._\- ]+[sS](?P<season>[0-9]{2})',                                                                                                     # (title) S01
                    '(?P<season>[0-9]{1,2})a? Stagione.*',                                                                                                                # ##a Stagione
                    '(?P<season>[0-9]{1,2}).*',                                                                                                                           # ##
                    '^.*([Ss]aga]|([Ss]tory )?[Aa][Rr][KkCc]).*$'                                                                                                         # Last entry in array, folder name droped but files kept: Story, Arc, Ark, Video
                  ]    
      for folder in reverse_path[:-1]:                 # remove root folder from test, [:-1] Doesn't thow errors but gives an empty list if items don't exist, might not be what you want in other cases
        for rx in SEASON_RX :                          # in anime, more specials folders than season folders, so doing it first
          if re.match(rx, folder, re.IGNORECASE):      # get season number but Skip last entry in seasons (skipped folders)
            reverse_path.remove(folder)                # Since iterating slice [:] or [:-1] doesn't hinder iteration. All ways to remove: reverse_path.pop(-1), reverse_path.remove(thing|array[0])
            if rx!=SEASON_RX[-1] and len(reverse_path)>=2 and folder==reverse_path[-2]:  season_folder_first = True
            break

It's 'Ark xxx' format in '^.([Ss]aga]|([Ss]tory )?[Aa][Rr][KkCc]).$' regular expression that gets trigered by the Ark in "Dark souls"...

If we replace 'for rx in SEASON_RX' by 'for rx in SEASON_RX[:-1]' that should work

ecto1a2003 commented 4 years ago

looks like that has worked!

ZeroQI commented 4 years ago

i re-used the season and transparent folder detection of the ASS scanner, but youtube has different structure from anime so didn't saw the bug coming. Will close that bug report for now

Thanks again

ZeroQI / YouTube-Agent.bundle

collections not being generated #38