search

meeb / tubesync

Syncs YouTube channels and playlists to a locally hosted media server
GNU Affero General Public License v3.0
1.96k stars 123 forks source link

Exclude Playlists when adding a channel #54

Open abb0r opened 3 years ago

abb0r commented 3 years ago

Hello, I just discovered Tubesync today and it works great so far. But I have a few channel that I added, that have playlists with videos from other channels and it seems Tubesync downloads these videos aswell. Is there a way to sync only videos that are uploaded to the channel I added?

meeb commented 3 years ago

I can possibly filter these out, can you paste some example channels with this behaviour.

abb0r commented 3 years ago

https://www.youtube.com/c/MrWissen2goGeschichte This channel has some playlists with videos from other channels.

meeb commented 3 years ago

Hm, after checking this channel it seems to be possible to exclude videos not uploaded to the original channel from sub-playlists on channels, but it may be substantially more work than I first suspected. I'll add this to a longer term roadmap to look into.

abb0r commented 3 years ago

Thanks for your fast answers. Definitely looking forward to this feature!

PhuriousGeorge commented 3 years ago

This was an issue in a previous version of youtube-dl. Depending on the version is used (I can't tell), it may already be resolved if you update.

meeb commented 3 years ago

TubeSync uses an up to date version of youtube-dl, currently as of writing that's 2021.02.10 (the bundled software versions are in the footer on the web interface). Thanks though, I'll check on the upstream tickets etc. on this to see if there's an easier way. I can store the uploader account ID at the channel level then compare it to the channel sub-items as their metadata is indexed to filter out media with a different uploader ID, but this would be pretty laborious and probably a bit buggy so if youtube-dl itself features a filter for this it would be much easier. Also thanks for the sponsorship @PhuriousGeorge much appreciated :)

PhuriousGeorge commented 3 years ago

The way I worked around it when this issue occurred with yt-dl was to append /videos to the end of the request URL. Would that perhaps be an easy (at least immediate) solution?

meeb commented 3 years ago

Internally TubeSync already does append /videos to the channel URLs for indexing since about :v0.6. I really don't think there's any more requests to YouTube I can cut down on and still maintain base level functionality, alas.

meeb commented 3 years ago

URLs for each source type are generated from these templates: https://github.com/meeb/tubesync/blob/main/tubesync/sync/models.py#L122

PhuriousGeorge commented 3 years ago

With that query, yt-dl shouldn't be pulling any playlist data unless explicitly entered as a playlist. I'd be more interested in the details of @abb0r's circumstances.

meeb commented 3 years ago

That query includes playlists included under the channel, and those playlists can include videos that were not originally uploaded to the original channel. The annoying part is TubeSync can't tell the original uploader ID until the metadata gets crawled per video, and also currently the channel ID (e.g. mychannelname is actually JiojieoOdJOIE inside YouTube) isn't stored in the database to compare it. In "flat" crawl mode (to reduce the requests to YouTube dramatically) I don't think the information is available to youtube-dl to do the filtering their end either so this will probably have to be an extended feature to add in TubeSync. Feel free to suggest youtube-dl flags though if I've missed any.