meeb / tubesync

Syncs YouTube channels and playlists to a locally hosted media server
GNU Affero General Public License v3.0
1.75k stars 117 forks source link

Guide on correct Plex setup? #45

Open OmgImAlexis opened 3 years ago

OmgImAlexis commented 3 years ago

Is there a recommended way to setup Plex for this?

Mainly I'm after the Plex library settings to use. For example which agent and scanner. Do we add the base directory or just video?

meeb commented 3 years ago

Alas there isn't a Plex YouTube media agent built in by default so you can either just add it as "Plex Video Files" and "Personal Media" agent, or you can try a couple of the third party Plex agents like https://github.com/JordyAlkema/Youtube-DL-Agent.bundle - however installing third party agents is out of scope of this project documentation. You'll likely have to update the output pattern to contain the video ID in square brackets to allow the third party agents to detect the videos.

OmgImAlexis commented 3 years ago

You'll likely have to update the output pattern to contain the video ID in square brackets to allow the third party agents to detect the videos.

About this. Is there a way to handle this without needing the agent to pull in the data? I'd much prefer the nfo contain as much as possible that way it'd still work if the video was taken down.

meeb commented 3 years ago

Yeah, you can have TubeSync write metadata to XBMC-style XML nfo files and use something like https://github.com/gboudreau/XBMCnfoMoviesImporter.bundle as the agent instead most likely. I've not personally tested this, but it looks like it should work.

OmgImAlexis commented 3 years ago

For those finding this in the future,XBMCnfoMoviesImporter doesn't work as it's only for movies and I'm using a tv show library for this.

meeb commented 3 years ago

Ah, how about https://github.com/gboudreau/XBMCnfoTVImporter.bundle ?

OmgImAlexis commented 3 years ago

Yeah my bad, found it the second I commented that. Trying it now.

meeb commented 3 years ago

TubeSync doesn't currently write "series" or "episode" information into the nfo files as they're not obviously available in the metadata returned by youtube-dl so I'm not sure how this would work. The nfo writer could hack in the channel name or something as the "series" but the episode numbers aren't available, perhaps the upload date might work, sort of.

meeb commented 3 years ago

For reference, the nfo generation code is at:

https://github.com/meeb/tubesync/blob/main/tubesync/sync/models.py#L1068

And example available metadata to map to the XML tree is:

https://github.com/meeb/tubesync/blob/main/tubesync/sync/testdata/metadata.json (season_number and episode_number seem to be pretty much universally unset for almost all videos)

Feel free to paste a URL to a YouTube video that may have an episode number and series number and I'll check the metadata contains it or not, it would be a trivial patch to add support for it if the metadata does set it properly.

OmgImAlexis commented 3 years ago

So ideally what Iā€™m wanting is to have whole channels archived with every upload being an episode and years being the seasons. The same goes for playlists.

Here's a channel I want https://www.youtube.com/c/TheHookUp

OmgImAlexis commented 3 years ago

The other option is absolute order using the "airdate". It just makes it hard to find an episode in 100s or 1000s of videos.

OmgImAlexis commented 3 years ago

Would also be good to have tubesync create a tvshow.nfo in the base directory so that Plex can use that for the show info.

meeb commented 3 years ago

The aired date is already set:

https://github.com/meeb/tubesync/blob/main/tubesync/sync/models.py#L1146

OmgImAlexis commented 3 years ago

The aired date is already set:

https://github.com/meeb/tubesync/blob/main/tubesync/sync/models.py#L1146

Okay perfect so I can use absolute for now.

undaunt commented 3 years ago

The playlist/series info is available as a separate .info.json file if you're using the youtube-dlp fork of youtube-dlc by pukkandan.

meeb commented 3 years ago

Thanks for the suggestion, I've taken a look at that fork and I don't think it provides any additional info that would be useful here although I'll certainly keep an eye on it for any useful new features.

There's no real "missing" playlist or series info other than series and episode style info. The issue is getting this info into Plex for this ticket which would require a media scanner agent to pick up all the info correctly which alas Plex won't do out of the box. TubeSync internally likely already stores all the required information (with the possible exception of something like #60). Media servers like Jellyfin should work out of the box with the nfo files TubeSync can write, but I've not gotten around to testing it or adding Jellyfin as a supported media server yet.

Longer term, short of manual Plex plug-in installation, TubeSync is likely to just spam videos into a general video folder without any extended Plex metadata support out of the box unfortunately.

extrobe commented 3 years ago

I'll add my 2cents worth, as I've now (almost) got this working exactly as I wanted, so thought I'd share...

1) Download media using TubeSync, ensuring you select the 'Create nfo file' and 'Get thumbnail' options 2) Install the XBMCnfoTVImporter agent as per the above instructions 3) In each channel folder created by TubeSync, create a text file named 'tvshow.nfo'

in this file, add the following info...

<tvshow> <title>Add Show Name Here</title> <season>1</season> <displayseason>1</displayseason> <displayepisode>1</displayepisode> </tvshow>

4) In Plex, add a new 'TV Programmes' library. Set the Scanner as 'Plex Series Scanner' and 'XBMCnfoTVImporter' as the agent. tick 'Disable agents artwork support (use local media assets instead)' I set 'Seasons' to 'Hide'

5) In Settings > Agents > Programmes > XMBCnfoTVImporter , tick 'Local media Assets (TV)' and drag to the top of the list

That's It. Not 100% sure 5) was necessary, but is how I have it configured. Plex will now import the show, with appropiate metadata and thumbnails. Only thing that's a little wonky is the episode ordering, but everything else is working well.

@meeb , if there was indeed an option to have the tvshow.nfo file created, it would be super useful, but it's fairly trivial to create it manually (happy to create a separate issue/request if this is something you'd be keen to look at)

meeb commented 3 years ago

Thanks for the guide @extrobe your testing is most useful. As for tvshow.nfo I think the reason one isn't currently written because it had to be at a directory level above in the plug-in or two I tested. As there are no "seasons" for the videos TubeSync downloads this didn't work. If I follow the XBMC guides correctly the tvshow.nfo should be:

/some-show/tvshow.nfo

With the media in

/some-show/Season 01/blah_s01eXX.ext

Or similar. Does tvshow.nfo work if it was just written into what is effectively the "season" directory?

extrobe commented 3 years ago

I just saved it to the root of the channel being created.

eg

/videos ---/channel-name ------/tvshow.nfo ------/video-1.mp4 ------/video-1.nfo ------/video-1.jpg ------/video-2.mp4 ------/video-2.nfo ------/video-2.jpg

I'm not using/creating season folders. In Plex, it does 'create' season folders based on Year, but I'm setting the library to not show season folders.

meeb commented 3 years ago

Edited: read up in the ticket, sorry was blind. Yep OK I can add this as a feature. Thanks for the feedback.

micahmo commented 3 years ago

Hey all,

If anyone is interested in an alternative that doesn't require TubeSync to generate any .nfos, I can share my setup. The caveat is that the scanner and agent must use the YouTube API for metadata, so if that's something you want to avoid, feel free to ignore this. :-)

  1. Download Absolute Series Scanner.py from ZeroQI/Absolute-Series-Scanner. Place it in Plex-Media-Server/Library/Application Support/Plex Media Server/Scanners/Series
  2. Download the codebase from ZeroQI/YouTube-Agent.bundle as a ZIP. Unzip and remove -master so you have a folder called YouTube-Agent.bundle. Place it in Plex-Media-Server/Library/Application Support/Plex Media Server/Plug-ins
  3. Add a new TV Shows library in Plex.
  4. Select the videos subfolder that TubeSync creates as the source.
  5. Choose Absolute Series Scanner as the scanner and YouTubeSeries as the agent. I also select "Set YouTube usernames as director in metadata" and I use "Episode" for Media Poster. Finally, be sure to input your own YouTube API key.
  6. Back in TubeSync, add a channel or playlist source. Set "Directory" to be the playlist or channel ID in brackets. If using a playlist, anything outside of the brackets will be ignored. If using a channel, the name before the brackets will be used as the channel name in Plex. Example: Kurzgesagt [UCsXVk37bltHxD1rDPwtNM8Q] for a channel or [PLFs4vir_WsTxontcYm5ctqp89cNBJKNrs] for a playlist. The agent formats playlists as a single season with episodes in chronological (not playlist added) order. The agent formats channels as date-based shows in Plex with a season per year. The agent picks the format based on the ID in brackets, so you could add a playlist in TubeSync and put the channel ID in the directory if you wanted the agent to format episodes by year seasons, for example.
  7. Finally, set the "Media format" to include the video ID (key) in brackets, like [{key}].{ext}. For channels, it's a good idea to add the upload time to the file format as well, like {yyyymmdd} [{key}].{ext}.
  8. With this setup, you do not need to select "Copy thumbnails" or "Write NFO".

I understand this won't fit everyone's requirements, but it might help someone. :-)

meeb commented 3 years ago

Interesting, thanks @micahmo - I can see how this would be useful for some, however I don't have a Google account and therefore no YouTube API keys to check this myself. If some other people can validate this indeed is reliable and works I'll add it as community documentation to the README. Cheers!

micahmo commented 3 years ago

No worries, and don't feel obligated to add it to the README. I get that many people use TubeSync specifically because they don't want/have a Google account. However, I'd hate to keep it to myself if it helps at least one person. :-)

BTW, the same instructions would apply if you wanted to use JordyAlkema/Youtube-DL-Agent.bundle, which you linked earlier in this thread. It also uses ZeroQI's scanner, but the agent doesn't use the Google API. It does require youtube-dl to be run with --add-metadata --write-info-json --write-thumbnail.

OmgImAlexis commented 3 years ago

@micahmo personally I can't use your setup as I want to archive content even after it's removed from Youtube so I'd need the metadata baked into or along side the files.

xtacle101 commented 3 years ago

@extrobe What are you using for your MEDIA FORMAT string? I'm following your steps and they all make sense to me, except for me, within Plex, it's pulling the "Title" as the date in YYYY-MM-DD format, and it's not using downloaded thumbnails.

my MEDIA FORMAT is currently {yyyy_mm_dd} - {title}_{format}.{ext} i figured it didn't matter, since the info that winds up in Plex should be coming from the nfo files.

Everything within the .nfo file looks good, and references the correct thumb file, but plex doesn't seem to be using the .nfo file.

thanks!

sinewave commented 2 years ago

@xtacle101, did you ever figure this out, I seem to be in the same boat as you...

  1. The title = the date
  2. It's not using the provided thumbnails but instead random preview thumbnails
sinewave commented 2 years ago

BTW, the same instructions would apply if you wanted to use JordyAlkema/Youtube-DL-Agent.bundle, which you linked earlier in this thread. It also uses ZeroQI's scanner, but the agent doesn't use the Google API. It does require youtube-dl to be run with --add-metadata --write-info-json --write-thumbnail.

@micahmo, @meeb, I'm also trying out this approach and wanted to ask if you knew how to add those flags and pass them to youtube-dl? I'm using TubeSync on Unraid using Docker.

meeb commented 2 years ago

--write-thumbnail is handled by TubeSync's "copy thumbnail" flag for the source. --write-info-json might be something I can add as a flag. --add-metadata unfortunately doesn't work properly with MKV containers at the moment and this is an upstream feature youtube-dl would need to add first. This is also holding off embedding thumbnails and some other nice-to-have features. I'm pondering switching to a youtube-dl fork which has more fancy features enabled by default. As for the direct flags passed to youtube-dl these are set here (note these are the API parameters not CLI --flags)

https://github.com/meeb/tubesync/blob/main/tubesync/tubesync/settings.py#L157

There's not an easy to use UI for these at the moment. You can copy the contents of local_settings.py, add in a YOUTUBE_DEFAULTS in it and then mount it back into the right location into the container as a volume for now if you really want to edit these, but obviously this is pretty non-user-friendly and you can break things if you get it wrong.

xtacle101 commented 2 years ago

I did get a setup that works:

for my "media format", i'm currently using: {source_full} - s{yyyy}e{mm}{dd} - {title_full} [{key}] {resolution}-WEBDL.{ext}

i have "Copy Thumbnails" and "Write NFO" both turned on.

this produces video files with a season/episode format with a year and date output of sYYYYeMMDD (ie s2021e0726) for my use cases, the channels i'm pulling don't release more than once per day, so this scheme has been working fine. not sure what the results would be if a channel had multiple videos per day. I imagine it would produce conflicts in Plex and merge videos together.

Within Plex, i have a library for the TubeSync content. Under it's advanced settings, i have: Scanner = Plex Series Scanner Agent = XBMCnfoTVImporter enable debug logging = CHECKED (for my testing - probably not necessary otherwise) disable agents artwork support (use local media assets instead) = CHECKED agents artwork location = local enable day first in ambiguous dates = UNCHECKED Seasons = Hide (this is the default setting - can override on a "per show" basis if needed, I just prefer it this way)

the rest of the XBMCnfoTVImporter settings i left untouched.

this produces a setup that works for me. it may not work for everyone or certain channels. hopefully this helps!

meeb commented 2 years ago

Thanks @xtacle101 very detailed and useful! Would a {minute} or similar macro for ouput file formatting be useful here to avoid day-level collisions?

cpt-kuesel commented 2 years ago

Could be helpful. I use S{yyyy}E{mm}{dd} - {title_full}.{ext} and for most cases everything works fine. But I have some channels with multiple videos a day. Most of the times the videos get merged, which is mildly annoying, but you can play versions and get to your content. Sometimes I get lucky and they are seperate.

S{yyyy}E{mm}{dd}{minute} should help get those videos seperated shouldn't it?

meeb commented 2 years ago

Yes it should separate videos in Plex. OK I'll add that as a feature.

cpt-kuesel commented 2 years ago

I experimented a bit with manually adding hour and minute to make the mask into sYYYYeMMDDhhmm, but Plex didn't recognise the files after this. Also a mask of sYYYYeMMDDhh didn't seem to get recognised by Plex either. #133 might not be the solution for multiple videos a day when using sYYYYeMMDD as a mask.

For my use case, keeping the videos only for a limited amount of time, I will be switching to sMMeDDhh once {hour} becomes available, as I don't intend to archive the videos, just want to watch them outside of youtube.

meeb commented 2 years ago

Yeah it looks like Plex might do a \d{1,4} or something on the regex for episode numbers so maxes out at 4 digits. I may look at also adding a per-source blind inrecementing counter macro as well which could be used as s{YYYY}e{counter} or similar, at least it would make Plex behave up to 9999 downloads per source (probably).

sinewave commented 2 years ago

Tku @xtacle101, that got it working for me šŸ‘šŸ‘šŸ‘

I may look at also adding a per-source blind inrecementing counter macro as well which could be used as s{YYYY}e{counter}

@meeb, this sounds like an excellent idea and something I would definitely use as it seems more inline with how episodes are released for the channels I follow. Using the date works ok, but can be confusing.

sinewave commented 2 years ago

Btw, has anyone figured a good automated solution to getting channel art/poster for Plex?

cpt-kuesel commented 2 years ago

i resorted to doing my own :/ pulling the channel pic and using it with an overlay from https://www.coverlabs.io/design for a low effort poster.

not perfect, but pretty quick once i got used to the workflow.

image

meeb commented 2 years ago

I did have a poke about at creating cover images automatically, but only the banners are available which didn't work at all. The thumbnails kind of work in some cases. There's nothing that's automatically the right size thought that doesn't look weird.

bwt615 commented 7 months ago

What's everyone doing for this nowadays?

I tried the zero tv series method to no success. +1 for native support of this somehow!