best way to ungroup episodes of a playlist

ecto1a2003 commented 3 years ago

I'm downloading https://www.youtube.com/playlist?list=PLiSdC6Lp8c7PGwbj9Ed0wYMMrz6ZU3rhQ

everything is showing as duplicate of episode 13 due to the playlist title containing the number 13 thoughts?

ZeroQI commented 3 years ago

No scanner logs nor confirmation you use ASS as scanner. No confirmation of series folder name nor files. Insufficient information provided

ecto1a2003 commented 3 years ago

ah, did realize you wantehd the whol shebang. yes im using ASS

dl string is youtube-dl -i -o "v:/internet/%(uploader)s/%(playlist_title)s [%(playlist_id)s]/%(title)s [%(id)s].%(ext)s" --download-archive "v:/internet/archive13.txt" --write-description --yes-playlist -f best https://www.youtube.com/playlist?list=PLiSdC6Lp8c7PGwbj9Ed0wYMMrz6ZU3rhQ

path is "V:\Internet\WoolieVersus\13 Sentinels - Aegis Rim [PLiSdC6Lp8c7PGwbj9Ed0wYMMrz6ZU3rhQ]\13 Sentinels - Aegis Rim (1) Prologue - Juro Kurabe [USh6Qutb3T8].mp4"

I can add logs soonish

ZeroQI commented 3 years ago

Numbering seem correct but if you only have couples bids in folder yes it will assign 13 as ep number although with youtube playlist ID it should number based on the playlist soon need the ASS scanner logs created per series in logs folder.

ecto1a2003 commented 3 years ago

WoolieVersus.scanner.log WoolieVersus.filelist.log

these are what your looking for?

also there seem to be several youtube folders in this directory should that be so?

ZeroQI commented 3 years ago

### Grouping folders skipped, will be handled by root level scan ### [return] Will need the root level scan logs since you use grouping folders (series folder not at library root level)

I need the log from the scanner showing the episode number translation, so I know why the playlist order is not used

A folder is created per library name in logs folder from memory.

ecto1a2003 commented 3 years ago

Well, instead of getting the logs I did something incredibly dumb or genius, I recreated the library and now I only have 200 videos in it versus the 800+ I had before. Logs will be uploaded in the morning.

ecto1a2003 commented 3 years ago

root.scanner.log root-share-CACHEDEV1_DATA-Videos-Internet.scanner.log root-share-CACHEDEV1_DATA-Videos-Internet.filelist.log

ecto1a2003 commented 3 years ago

WoolieVersus.filelist.log WoolieVersus.scanner.log

ecto1a2003 commented 3 years ago

com.plexapp.agents.youtube.log

com.plexapp.agents.youtube.log com.plexapp.agents.youtube1.log com.plexapp.agents.youtube2.log

ZeroQI commented 3 years ago

Can you try to include the channel tag on grouping folder? If it doesn't solve, put the folder with the playlist is at the root of the library folder, or create a test library pointing at the parent folder of the one With the playlist is...

ecto1a2003 commented 3 years ago

adding the channel tag to the grouping folder seems to have helped, the episodes are seperate but he name of the 'show' or playlist is now the path to the grouping folder.

/share/CACHEDEV1_DATA/Videos/Internet/WoolieVersus instead of 13 Sentinels: Aegis Rim

also im still missing about 900 videos from the library. Plex Media Scanner.log Plex Media Server.log root.scanner.log WoolieVersus [UCyOJzQzyDGihKpTO3-zyhYg].filelist.log WoolieVersus [UCyOJzQzyDGihKpTO3-zyhYg].scanner.log root-share-CACHEDEV1_DATA-Videos-Internet.scanner.log root-share-CACHEDEV1_DATA-Videos-Internet.filelist.log

ecto1a2003 commented 3 years ago

com.plexapp.agents.youtube1.log com.plexapp.agents.youtube.log

ecto1a2003 commented 3 years ago

show name fixed with update to newest agent version

ZeroQI commented 3 years ago

Videos missing are a scanner issue

Changing the folder name will not rescan folder usually except you use grouping folders which rescan every time (no cache), and only supported with my scanner
ASS has been updated as well? You have loads of mp4.part files hope you are not counting those

/share/CACHEDEV1_DATA/Videos/Internet/WoolieVersus [UCyOJzQzyDGihKpTO3-zyhYg]/13 Sentinels - Aegis Rim [PLiSdC6Lp8c7PGwbj9Ed0wYMMrz6ZU3rhQ]

looking at "WoolieVersus [UCyOJzQzyDGihKpTO3-zyhYg].scanner.log", you have proper season and episode number...

Please update all, pick up one playlist with issues, attach the xxx.scanner.log file for it and give me the issue...

ecto1a2003 commented 3 years ago

The other interesting thing here is if I make a library and point it at a channel, woolie VS for instance everything shows up perfectly and is named and numbered correctly. The root library is the weird issue, I have multiple channels that dont even show up an I am ignoring the. Part files. I'll upload more logs in the am.

ecto1a2003 commented 3 years ago

It also seems that the show title is the Chanel title instead of the Playlist title can that be changed?

ecto1a2003 commented 3 years ago

Super Best Friends Play [UC84X0epDRFdTrybxEX8ZWkA].filelist.log Super Best Friends Play [UC84X0epDRFdTrybxEX8ZWkA].scanner.log Super Best Friends Play [UC84X0epDRFdTrybxEX8ZWkA].scanner.log

here is a channel that has multiple playlist inside. none show up.

ecto1a2003 commented 3 years ago

WoolieVersus.scanner.log I moved the library form one folder to another without renaming and a lot of my videos are populating again.

any folder that had the channel id wouldn't show up so I removed it and now the 13 sentinels are grouped as one episode now. the logs say it registers as anime instead of youtube

ecto1a2003 commented 3 years ago

circling back, heres another example of episodes showing and duplicate when they shouldn't 2020-12-21 10_38_28-Start

WoolieVersus [UCyOJzQzyDGihKpTO3-zyhYg].scanner.log WoolieVersus [UCyOJzQzyDGihKpTO3-zyhYg].filelist.log com.plexapp.agents.youtube.log com.plexapp.agents.youtube2.log com.plexapp.agents.youtube3.log com.plexapp.agents.youtube4.log com.plexapp.agents.youtube5.log com.plexapp.agents.youtube1.log

ecto1a2003 commented 3 years ago

I played around with different grouping rules and naming and I think I have it how I like it now.

reddragonguy commented 3 years ago

@ecto1a2003 can you post your naming scheme. I'm struggling with this right now myself.

ecto1a2003 commented 3 years ago

@reddragonguy here's what i have. if you get any weirdness you should just need to fix match on each serires/playlist

youtube-dl -ciw -o "v:/youtube/%(channel)s/%(playlist_title)s [%(playlist_id)s]/%(title)s [%(id)s].%(ext)s"

reddragonguy commented 3 years ago

@ecto1a2003 thanks for this. Although, i've been using uploader instead of channel as that generally ends up being "NA". Another question, are you using playlist-end or playlist-start in order to limit the number of videos being inspected per call? I have one playlist that has 5K+ videos in it and it takes hours to run. I've tried using --dateafter now-30days along with --playlist-end 30 and --playlist-reverse to shorten the time with no luck.

ecto1a2003 commented 3 years ago

@reddragonguy good catch, that is the correct thing to use. my bad. I've never used those flags before, but i think it should be now-30day ?

ecto1a2003 commented 3 years ago

nope scratch that, reading is hard.

ecto1a2003 commented 3 years ago

i think im having this issue again, should i reopen this issue or open another one?

ZeroQI commented 3 years ago

Well you reopened it, but latest post before closing says you changed the naming and were happy enough. so why do you think you have this issue again? If it's a numvering issue you need to provide ASS scanner log

ecto1a2003 commented 3 years ago

I've migrated to a new installation. I have a few different issues currently. The biggest is that episodes don't show up at all, after that episodes are merged again even though they have nothing in common. and lastly seasons are being created based on file name - ideally there'd only be the one season per playlist, i.e season 1.

PeachSaliva [UCMaikIXU5cF6nS2YgJPYQsQ].filelist.log PeachSaliva [UCMaikIXU5cF6nS2YgJPYQsQ].scanner.log PlagueOfGripes [UC0PaoSxEK2nEmO-Mtk2vGWA].filelist.log

PlagueOfGripes [UC0PaoSxEK2nEmO-Mtk2vGWA].scanner.log root-share-CACHEDEV1_DATA-Videos-testyoutube.filelist.log root-share-CACHEDEV1_DATA-Videos-testyoutube.scanner.log Pat Stares At [UCpmNJI_KhjkpXw1UPmtC3-Q].filelist.log Pat Stares At [UCpmNJI_KhjkpXw1UPmtC3-Q].scanner.log com.plexapp.agents.youtube.log

%(uploader)s [%(channel_id)s]/%(playlist_title)s [%(playlist_id)s]/%(title)s [%(id)s].%(ext)s

ZeroQI commented 3 years ago

ASS readme first lines reads:

A Plex Series Scanner makes the video files showing in Plex and populate the following for the video files: -Series name -Series year -Season number -Episode number -Episode title (not filled by plex default series scanner, but ASS fills it, but this will be overwritten by the metadata agent)

If a file is not showing in plex or showing at the wrong season and/or episode number, or is not passing through the forced id, then it is a scanner issue. Anything else is metadata related and agent specific.

The biggest is that episodes don't show up at all => That's a scanner issue... after that episodes are merged again => That's a scanner issue... seasons are being created based on file name => That's a scanner issue...

I will move this issue to ASS

Extract from logs with impacted eps:

Library: 'moretesting', root: '/share/CACHEDEV1_DATA/Videos/testyoutube', path: 'Pat Stares At [UCpmNJI_KhjkpXw1UPmtC3-Q]/Dark Souls 2 SotFS [PLPXYESFhUD6Rkvq2XWd6HYgrCXaxFa8hC]', dirs: '[]'
"Pat Stares At [UCpmNJI_KhjkpXw1UPmtC3-Q]" s2018e002                         "YouTube" "2 - Pat Streams At Borderlands 2 (Part 2) 2017-08-05 [0WJyRNOXIsM].mp4" "2 - Pat Streams At Borderlands 2 (Part 2) 2017-08-05 [0WJyRNOXIsM].mp4"

Library: 'moretesting', root: '/share/CACHEDEV1_DATA/Videos/testyoutube', path: 'Pat Stares At [UCpmNJI_KhjkpXw1UPmtC3-Q]/Dark Souls 3! [PLPXYESFhUD6TngQhVw5bYxZJgedPjYYB0]', dirs: '[]'
"Pat Stares At [UCpmNJI_KhjkpXw1UPmtC3-Q]" s2018e002                         "YouTube" "2 - Pat Stares At Dark Souls 3 (Part 2) 2018-11-22 [g9yqrtGtQjc].mp4" "2 - Pat Stares At Dark Souls 3 (Part 2) 2018-11-22 [g9yqrtGtQjc].mp4"

Series name: "Pat Stares At [UCpmNJI_KhjkpXw1UPmtC3-Q]" and Numbering: s2018e002 so Logical it does group them

I understand the playlists in the channel folder, but with this makes it with the channel id a forced id channel instead of the playlist. reproducing but think it should be a series named after the playlist in season 1 and use the collection field from grouping folder to group them in a collection per channel

ZeroQI commented 3 years ago

Please let me know if fixed as expected The playlist folder if ending with an 's' before a number was seen as a season folder and removed, thus leaving the channel folder as series folder, of course season folders are removed before logging starts...

ecto1a2003 commented 3 years ago

Pat Stares At [UCpmNJI_KhjkpXw1UPmtC3-Q].scanner.log PeachSaliva [UCMaikIXU5cF6nS2YgJPYQsQ].filelist.log PeachSaliva [UCMaikIXU5cF6nS2YgJPYQsQ].scanner.log PlagueOfGripes [UC0PaoSxEK2nEmO-Mtk2vGWA].filelist.log PlagueOfGripes [UC0PaoSxEK2nEmO-Mtk2vGWA].scanner.log root-share-CACHEDEV1_DATA-Videos-testyoutube.filelist.log root-share-CACHEDEV1_DATA-Videos-testyoutube.scanner.log Pat Stares At [UCpmNJI_KhjkpXw1UPmtC3-Q].filelist.log

there's nothing now

ZeroQI commented 3 years ago

Did you remove all files from library, rescanned, cleaned bundles, put them back, rescanned (aka Plex dance)? If nothing is scanned, then the scanner crashed and Plex Media Scanner.log for the error

Generated library file [https://github.com/ZeroQI/Hama.bundle/wiki/Batch-file-to-create-a-relative-list-of-files-including-subfolders] would allow me to recreate your library in one go to avoid back and forth

ecto1a2003 commented 3 years ago

look slike it just needed time, thngs are better but theres still duplicates. filelist.txt

PeachSaliva.filelist.log PeachSaliva.scanner.log PlagueOfGripes.filelist.log PlagueOfGripes.scanner.log WoolieVersus.filelist.log WoolieVersus.scanner.log root-share-CACHEDEV1_DATA-Videos-testyoutube.filelist.log root-share-CACHEDEV1_DATA-Videos-testyoutube.scanner.log Pat Stares At.filelist.log Pat Stares At.scanner.log

ZeroQI commented 3 years ago

The scanner uses the rank of the video in the chan video list and did give it different episode numbers:

"Uploads from PeachSaliva [UUMaikIXU5cF6nS2YgJPYQsQ]" s01e067                         "YouTube" "Ruby Weapon Hour - DELTARUNE 2018-10-31 Spooky Halloween!" "Ruby Weapon Hour - DELTARUNE 2018-10-31 Spooky Halloween! [dNIhDcV-BwU].mp4"
[BOSom83e6js] rank:  26 in file: Ruby Weapon Hour - Phoenix Wright - Ace Attorney! (Part 2) [BOSom83e6js].mp4
"Uploads from PeachSaliva [UUMaikIXU5cF6nS2YgJPYQsQ]" s01e065                         "YouTube" "Ruby Weapon Hour - Hangin' out + Fanfic 2018-11-19" "Ruby Weapon Hour - Hangin' out + Fanfic 2018-11-19 [LWVjiJ34d1M].mp4"
[fX98lIfc6Ak] rank: 139 in file: Ruby Weapon Hour - Bayonetta (Part 2) 2017-05-13 [fX98lIfc6Ak].mp4
"Uploads from PeachSaliva [UUMaikIXU5cF6nS2YgJPYQsQ]" s01e085                         "YouTube" "Ruby Weapon Hour - Heavy Rain (Part 2) 2018-04-23" "Ruby Weapon Hour - Heavy Rain (Part 2) 2018-04-23 [eXzfic67fhk].mp4"
[n3hGd5YVvn8] rank: 100 in file: Ruby Weapon Hour - Diablo 3 w_ Mr Clemps and Volta Bass! 2018-02-19 [n3hGd5YVvn8].mp4

What episode number is your example ?

ecto1a2003 commented 3 years ago

plex shows it as episode 12 - episode 5 , the duplication is across multiple shows, except they all seem to be in the default uploads folder. they should still not combine from that. or is it possible for those videos to join the playlists they belong to without being sorted correctly?

ecto1a2003 commented 3 years ago

here's another example that isn't in the default uploads folder

ZeroQI commented 3 years ago

reproduced your library and don't have that problem, so you probably have an old ass version (solved 3 days ago) Update, remove the folders from the library, rescan, delete bundles, add the folders back, rescan, it should work

ecto1a2003 commented 3 years ago

hmm, i have the latest ASS. let me reinstall my plex as its been acting funny.

ecto1a2003 commented 3 years ago

still 0 content after rebuild. Pat Stares At.scanner.log PeachSaliva.filelist.log PeachSaliva.scanner.log PlagueOfGripes.filelist.log PlagueOfGripes.scanner.log WoolieVersus.filelist.log WoolieVersus.scanner.log root-share-CACHEDEV1_DATA-Videos-testyoutube.filelist.log root-share-CACHEDEV1_DATA-Videos-testyoutube.scanner.log Pat Stares At.filelist.log

ecto1a2003 commented 3 years ago

seems to be related to a similar issue from before. if there's more than one playlist in a grouping folder it gets dropped.

ZeroQI commented 3 years ago

Please test latest release I still had your folders in my test library and it detected OK. Fully rewrote logging, each file has ONE copy of logs, root one has list of all folders and if grouping folder, season folder, etc...

In case of issues, root.scanner.log and sereis filelist and scanner.log.

ecto1a2003 commented 3 years ago

i dont get it...

still empty.

Pat Stares At.scanner.log PeachSaliva.filelist.log PeachSaliva.scanner.log PlagueOfGripes.filelist.log PlagueOfGripes.scanner.log WoolieVersus.filelist.log WoolieVersus.scanner.log

ecto1a2003 commented 3 years ago

root.scanner.log

root.filelist.log is empty

ecto1a2003 commented 3 years ago

filelist.txt

ZeroQI commented 3 years ago

Please post 'Plex Media Scanner.log Pat and PeachSaliva look grand at least.

ecto1a2003 commented 3 years ago

it looks like it thinks the season folders are grouping folders

ecto1a2003 commented 3 years ago

Plex Media Scanner.log

ZeroQI commented 3 years ago

Here's a big crash at the beginning line 840,well that explains everything...

Mar 05, 2021 15:27:59.239 [0x7ff30a4e2780] ERROR - Error in Python: Running scanner: Traceback (most recent call last): File "/share/CACHEDEV1_DATA/.qpkg/PlexMediaServer/Library/Plex Media Server/Scanners/Series/Absolute Series Scanner.py", line 1102, in Scan Scan(path, sorted(subdir_files), media, sorted(subdir_dirs), language=language, root=root, kwargs_trigger=True) #relative path for dir or it will show only grouping folder series File "/share/CACHEDEV1_DATA/.qpkg/PlexMediaServer/Library/Plex Media Server/Scanners/Series/Absolute Series Scanner.py", line 840, in Scan if VideoID and VideoID in file.decode('utf-8'): File "/share/CACHEDEV1_DATA/.qpkg/PlexMediaServer/Resources/Python/lib/python2.7/encodings/utf_8.py", line 16, in decode return codecs.utf_8_decode(input, errors, True) UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 9: ordinal not in range(128) Mar 05, 2021 15:27:59.239 [0x7ff30a4e2780] ERROR - We got an error scanning in /share/CACHEDEV1_DATA/Videos/testyoutube

Replace line 840 with the first and test and the second if it doesn't work

if VideoID and VideoID in file:
if VideoID and VideoID in sanitize_path(file):

It will solve grouping folder series like youtube but normal cached folder whose series folder is at root level will need the plex dance

Sent from my bath at midnight here so you can understand I cannot reproduce now

ecto1a2003 commented 3 years ago

tried both no success.

Plex Media Scanner.log

ZeroQI commented 3 years ago

The legend for the table is not perfect, the grouping folder being at library root is scanned by plex normal call while the series folder inside are scanned by a scan() call while in root folder, it should be Root call to be clearer... Edited the code for clarity.

There is no crash in the logs but it is scanning using Plex scanner "Performing a scan with 'Plex Series Scanner' (language: en virtual: 0)."so a scan ran after youtube library scan, so reproduced your library and this comma in the filename gave me headaches: PlagueOfGripes\Animations [PLPBmbs-JPhHc-__sPdp3seXz-MWOVJP07]\37 - BLUD‚mon [Vn05qpn2e1Y].mp4`

i have the show and episode title as UTF-8 as required by Plex, but all strings showing as unicode and this was converted to string and crashing... Now working with your library on my system

ZeroQI / Absolute-Series-Scanner

best way to ungroup episodes of a playlist #338