Collection Groupings - Githubissues

djmixman commented 6 years ago

Collections are not being properly grouped when sub-folders of two "Collection Group" folders contain the same name.

Example:

H2ODelirious/Ben and Ed CaRtOoNz/Ben and Ed

will be grouped together into the same collection of either H2O or Cartoonz...

ZeroQI commented 6 years ago

if there is 2 folder depth or more compared to the library root folder (relative path), the folder containing the series folder is the grouping folder and its named used for a collection

      if len(reverse_path)>1 and not season_folder_first and subfolder_count>1:  ### grouping folders only ###
        Log.Info("Grouping folder found, root: {}, path: {}, Grouping folder: {}, subdirs: {}, reverse_path: {}".format(root, path, os.path.basename(series_root_folder), subfolder_count, reverse_path))
        Log.Info('[ ] collections:        "{}"'.format(reverse_path[-1]))
        if reverse_path[-1] not in metadata.collections:  metadata.collections=[reverse_path[-1]]

      else:  Log.Info("Grouping folder not found or single folder, root: {}, path: {}, Grouping folder: {}, subdirs: {}, reverse_path: {}".format(root, path, os.path.basename(series_root_folder), subfolder_count, reverse_path))

I did reproduce and i see one series name with no channel/playlist id therefore having its unique identifier being the folder name (assigned by search() function) merge with another folder named not only identically but with the same unique id.

Can you give a channel id to the sries name 'Ben and Ed [UCxxxxxxxxxxxx]' to either folder and confirm it beahaves as expected Collections are not removed so you need to remove it from series if needed

djmixman commented 6 years ago

Directory Structure:

./CaRtOoNz/Ben and Ed [UCdQWs2nw6w77Rw0t-37a4OA]:
    20160113 - Ben and Ed _ 'My Zombie Best Friend!' (I Didn't Need Those Legs Anyway!) [fRFr7L_qgEo].mkv
    20160116 - Ben and Ed _ 'Clownin Around!' (F_ck You Neck-Beard!) [Nh9eILgD5N4].mkv

./CaRtOoNz/Golf With Your Friends [UCdQWs2nw6w77Rw0t-37a4OA]:
    20160710 - Golf With Friends _ HOLE IN ONE...THOUSAND! (w_ H2O Delirious, Bryce, & Ohmwrecker) [81er8CP24h8].mkv
    20160713 - Golf With Friends _ GOLF LIKE AN EGYPTIAN! (w_ H2O Delirious, Bryce, & Ohmwrecker) [gKYid-SjDiE].mkv

./H2ODelirious/Ben and Ed [UCClNRixXlagwAd--5MwJKCw]:
    20160104 - Ben And Ed Ep.1 (MUST SAVE BEN) BRAINNNNNSSSS [9YeXl28l9Yg].mkv
    20170721 - Ben And Ed - Blood Party - ANGRYLIRIOUS!!!!! (I CAN DO THIS!) [BEDE2z3G3hY].mkv

./H2ODelirious/Golf With Your Friends [UCClNRixXlagwAd--5MwJKCw]:
    20160711 - Golf With Your Friends - 1st Time Playing! 'Professionals' [wxS52xI_W_Y].mkv
    20160715 - Golf With Your Friends - Hitting Balls, Stroking Out! [GdLon0CCEXE].mkv

This groups "Ben and Ed [\]" into each collection, but it also combines Golf With Friends in the same show.

So we have a two part issue here:

1a: Everything within the "Channel" folder is grouped into the first "Show" folder.

1b: All the logs show a unknown url error:

exception: <urlopen error unknown url type: unknown id>, url: https://www.googleapis.com/youtube/v3/playlistItems?part=snippet&maxResults=50&playlistId=UCdQWs2nw6w77Rw0t-37a4OA&key=AIzaSyC2q8yjciNdlYRNdvwbb7NEcDxBkv1Cass
iteration: 1, nextPageToken: "", url: Unknown id: UCdQWs2nw6w77Rw0t-37a4OA

2: If you remove the [Channel ID] from the custom playlist folder it seems to mix and match them however it pleases.

ZeroQI commented 6 years ago

1a. If two series|movies are going to use the same unique ID (GUID|metadata.id), Plex will merge them together

1b. Channel id is valid: https://www.youtube.com/channel/UCdQWs2nw6w77Rw0t-37a4OA URL use 'playlistItems' so YOUTUBE_PLAYLIST_ITEMS but url loadin is failing saying 'Unknown id: UCdQWs2nw6w77Rw0t-37a4OA' but normal as it shows a channel ID in the url

error in scanner code, in progress

2a. We need truly unique IDs

if a channel|playlist id exist i can use that but not if two folders are going to use it, Plex will merge them so i need to make the GUIDfor series with channel id 'Channel_ID-Folder_name' to make it unique
playlist will not be duplicated so the GUID can stay "Playlist_ID" (unless people start splitting playlists)
for the same reason cannot use the folder name alone as it will merge series with the same unique ID so would help to do 'grouping_folder-folder_name' if in a grouping_folder else 'folder_name'

if i could support channel id in grouping (collection) folder to use for series folder would be great

ZeroQI commented 6 years ago

Progress

i have collection created and eries inside automatically
i was right, if search() assign the same GUID/Ubique id twice, you can never separate series again afterwards
posters from first video
removed channel items since limited to 1000 items
added summary w/ # videos, # views, #subscriptions if no channel summary
you can put channel id on grouping folder now and it cascade in the title automatically
channel id make scanner use year of file as season and sort files from oldest to newest

Uploaded new agent and scanner, this one should be nice

Two half days of coding to solve...
anybody need to re-match all
i did reproduce your structure but with channel id on grouping folder

Will have to modify read-me but i lack the mental fortitude, will start building house soon with brother in law as foreman, will have even less time soon

Here is my test folder

CaRtOoNz [UCdQWs2nw6w77Rw0t-37a4OA]
- Ben and Ed:
    Ben and Ed _ 'My Zombie Best Friend!' (I Didn't Need Those Legs Anyway!) [fRFr7L_qgEo].mkv
    Ben and Ed _ 'Clownin Around!' (F_ck You Neck-Beard!) [Nh9eILgD5N4].mkv
- Golf With Your Friends
    Golf With Friends _ HOLE IN ONE...THOUSAND! (w_ H2O Delirious, Bryce, & Ohmwrecker) [81er8CP24h8].mkv
    Golf With Friends _ GOLF LIKE AN EGYPTIAN! (w_ H2O Delirious, Bryce, & Ohmwrecker) [gKYid-SjDiE].mkv

/H2ODelirious [UCClNRixXlagwAd--5MwJKCw]
- Ben and Ed 
    Ben And Ed Ep.1 (MUST SAVE BEN) BRAINNNNNSSSS [9YeXl28l9Yg].mkv
    Ben And Ed - Blood Party - ANGRYLIRIOUS!!!!! (I CAN DO THIS!) [BEDE2z3G3hY].mkv
- Golf With Your Friends
    Golf With Your Friends - 1st Time Playing! 'Professionals' [wxS52xI_W_Y].mkv
    Golf With Your Friends - Hitting Balls, Stroking Out! [GdLon0CCEXE].mkv

Please test and report

djmixman commented 6 years ago

Right now i'm getting this error:

2018-06-25 22:15:50,531 (805bd3400) :  INFO (__init__:106) - =============================================================================================================================================================
2018-06-25 22:15:50,531 (805bd3400) :  INFO (__init__:107) - search()
2018-06-25 22:15:50,531 (805bd3400) :  CRITICAL (agentkit:1014) - Exception in the search function of agent named 'YouTube', called with keyword arguments {'openSubtitlesHash': '7584b0b768135057', 'episode': '2', 'name': "20160711 - Golf With Your Friends - 1st Time Playing! 'Professionals' [wxS52xI_W_Y].mkv", 'episodic': '1', 'show': 'Golf With Your Friends [UCClNRixXlagwAd--5MwJKCw] [UCClNRixXlagwAd--5MwJKCw]', 'season': '2016', 'plexHash': 'b0df406217b8a7ca994496dde141e939ba478b65', 'filename': "%2Fmnt%2FMedia%2FVideos%2FTestTube4%2FH2ODelirious%2FGolf%20With%20Your%20Friends%20[UCClNRixXlagwAd--5MwJKCw]%2F20160711%20-%20Golf%20With%20Your%20Friends%20-%201st%20Time%20Playing!%20'Professionals'%20[wxS52xI_W_Y]%2Emkv", 'year': '2016', 'duration': '-1', 'id': '71104'} (most recent call last):
  File "/usr/local/share/plexmediaserver/Resources/Plug-ins-fd05be322/Framework.bundle/Contents/Resources/Versions/2/Python/Framework/api/agentkit.py", line 1007, in _search
    agent.search(*f_args, **f_kwargs)
  File "/usr/local/plexdata/Plex Media Server/Plug-ins/YouTube-Agent.bundle/Contents/Code/__init__.py", line 433, in search
    def search (self, results,  media, lang, manual):  Search (results,  media, lang, manual, False)
  File "/usr/local/plexdata/Plex Media Server/Plug-ins/YouTube-Agent.bundle/Contents/Code/__init__.py", line 143, in Search
    Log('search() - id not found')
UnboundLocalError: local variable 'regex' referenced before assignment

I'll try to debug in a few, but I might not be able to get around to it tonight...

ZeroQI commented 6 years ago

uploaded fix My code was working on my code from before your big submit but including it in you code fix was done line by line in notepad++ and missed one, sorry...

djmixman commented 6 years ago

So far everything seems to be working... Everything seems to be grouping correctly and playlists (with channel ids) sorts into the correct folders, now one other hopefully small issue I am seeing.

When it adds the "show" it will also include the [channelid] in the name:

It seems if you go in and edit the show name and remove the [channelid] it will work and not conflict with the other show that's named the same.

I'm not sure if this will pose a problem later though.

ZeroQI commented 6 years ago

@djmixman it won't, the guid being already assigned but would be over-written possibly unless locked corrected it quickly

You can move the channel id to the parent folder btw...
I wish i could not crop the screen capture but cannot find a way

Functions wise, i think the scanner and agents are good and working together. Glad it works at last. Chyannel mode is good and use file age to number in order as advised, simple and effective

djmixman commented 6 years ago

Nice :) 👍 It is all coming together great!

As for the artwork:

I'll see if I can work out a nice solution to fix the artwork later on (maybe this weekend) but I doubt there is a decent way since they are two different aspect ratios.

When you get down to episode level it looks great!

but anything beyond that and you get the cropping issues. :( Maybe someone can chime in with a solution. Should we create a new ticket for this specific issue?

djmixman commented 6 years ago

I think this can probably be closed for now since the original issues appears to be fixed... Will reopen is other issues are found.

ZeroQI commented 6 years ago

i would prefer the black bars or doubling verticaly the screenshot ot putting black bars top and bottom to get the proper ratio... I tried finding image libraries to import in python but couldn't find any but am a self-taught novice in python, i thing i did well considering but not many donations for youtube agent :/

djmixman commented 6 years ago

Its a very niche market so I wouldn't expect much at all. I plan to send over a little, but it wont be much at all. Just gotta wait for payday.

I'm no master at python myself but i've dabbed enough in it to kind of know how to get something done. Usually if I get a start, like with your code, I can go wild. But trying to figure out the start leaves be dumbfounded.

ok, getting off a little too off topic now :P

ZeroQI commented 6 years ago

we sure do... i started with a crappy hama agent that was anidb only and now it's a gas factory... i then created the scanner from scratch since i couldn't understand how the official worked too much and ended up allowing grouping folders, so we can put series in subfolders compared to the library root... That in turn enabled youtube agent and automatic collections and what i believe is the best ecosystem for anime and youtube now... although yeah the cropping in the series poster is annoying... Gotta go sleep, 1am here...

ZeroQI / YouTube-Agent.bundle

Collection Groupings #3