search

bibanon / tubeup

Use yt-dlp to download video/metadata and upload to the Internet Archive.
https://pypi.python.org/pypi/tubeup/
GNU General Public License v3.0
424 stars 71 forks source link

Add flag --no-dlp or --use-youtubedl #222

Closed upintheairsheep closed 2 years ago

upintheairsheep commented 2 years ago

I recently bought an iPhone 13 Noob (128GB, Black). All the shell apps cannot install yt-dlp due to the library used for encryption uses C, which iOS doesn't support. YouTube-DL works on the other end. Some other operating systems or specific computers may have yt dlp fail installation or just be really slow compared to youtube dl. There are also some features, fixes, etc available on YouTube-DL but not yt-dlp like for example, issues exclusive for yt dlp or support for deepfake porn sites or sites not yet added to dlp. This also might be for future proofing if yt dlp gets discontinued or YouTube dl gets more features.

brandongalbraith commented 2 years ago

@upintheairsheep I'm not opposed to making the extractor pluggable with the ability to specify which extractor to use with a command line option, but you'll need to submit a PR for this feature if desired.

vxbinaca commented 2 years ago

I'm opposed to adding in unreliable downloaders to tubeup to make one single user be able to rip videos. Youtube-dl are slow to implement fixes, yt-dlp is better in every single way. YTDL If you want yt-dlp to drop the crypto requirement then ask them to change. we've now added flags and behavior for ourselves and others that is incompatible with YTDL. We'd have a schizophenic setup where most flags work but some don't.

Everyone wants us to change, to adapt to their extremely specific needs. At some point a foot needs to be put down.

@upintheairsheep it's time for you to buy a cheap linux VPS with enough storage (at least 50GB), that are $6 a month. Do your rips there not on a iphone. And if you want comments from TikTok to be extracted you need to talk to the yt-dlp people.

vxbinaca commented 2 years ago

Again if any of you think I'm being unfair, Twitch extractor was broken for 6 months on YTDL and their maintainers refused to merge the code despite it passing checks and it was tested by others in the community. I had 3 months where I couldn't do any rips of Twitch for months because their negligent staff weren't pushing up fixes or releasing new versions. Absolutely not ever going back to YTDL.

I prize stability over one persons unique situation.

It's time to get a VPS upintheair. They're cheap and with enough disk space can do most channels out there. Bite the bullet and get a VPS it's strongly recommended you not do channel rips from your desktop.

brandongalbraith commented 2 years ago

I don't think you're being unfair Paul. Maintaining open source tooling is not easy work, and our work capacity is limited (look how long it took to get the IA creds bug fixed until someone else could step in and get to it because I didn't have the time). Really want to be open to new ideas and code for the tooling, but sometimes it's going to be easier for folks to fork us and maintain a version that more closely meets their use/edge case.

vxbinaca commented 2 years ago

so YTDL uses multiple downloaders, but our downloader (YT-DLP) has to work the first time every time thats why we do the configuration for the user, also so IA has some consistency to expect. If the extractors for say, imgur, drift between YTDL and DLP, where YTDLs has a bug that their devs sit on their hands for a while to fix but works in DLP, and we have a configuration option to allow users to switch downloaders and then some people do YTDL and some do DLP, then Archive.org may not be reliably able to reliably correct for the problem based on the range of when the items were made due to some users using a different downloader.

I would rather have DLP have a problem that they'll fix in a reasonable amount of time, so theres the shortest downtime, then a schizophrenic situation where some people pick downloaders. Not to mention maintaining the lib call compatibility will suck because slowly DLP is drifting in it's names for library calls.

I can already see it being a issue in the future and it's driving me up a wall thinking about the potential loss that would occur.

I'm really trying not to be unfair and a jerk here but this is a problem I can see cropping up.

Lemme talk to Puka about the encryption dep and why he needs it.

vxbinaca commented 2 years ago

There are also some features, fixes, etc available on YouTube-DL

Name one. They pull code frequently from YTDL into DLP.

issues exclusive for yt dlp or support for deepfake porn sites or sites not yet added to dlp.

Archive.org ain't gonna like that very much and I'd suggest you not do that.

This also might be for future proofing if yt dlp gets discontinued or YouTube dl gets more features.

I think the opposite will happen and YTDL will cease development first. But say you're right, then I'll switch back. I'd have to clean up the lib calls and deps but thats about it

upintheairsheep commented 2 years ago

OK. I’ll just use an older version. I’m broke in reality and don’t have a credit card yet. I’ll use an old version instead or use my sister’s old Samsung A02s she doesn’t care about since she got her iPhone 13 Noob.

On Fri, Jun 24, 2022 at 4:23 PM Paul Henning @.***> wrote:

I'm opposed to adding in unreliable downloaders to tubeup to make one single user be able to rip videos. Youtube-dl are slow to implement fixes, yt-dlp is better in every single way. YTDL If you want yt-dlp to drop the crypto requirement then ask them to change. we've now added flags and behavior for ourselves and others that is incompatible with YTDL. We'd have a schizophenic setup where most flags work but some don't.

Everyone wants us to change, to adapt to their extremely specific needs. At some point a foot needs to be put down.

@upintheairsheep https://github.com/upintheairsheep it's time for you to buy a cheap linux VPS with enough storage (at least 50GB), that are $6 a month. Do your rips there not on a iphone. And if you want comments from TikTok to be extracted you need to talk to the yt-dlp people.

— Reply to this email directly, view it on GitHub https://github.com/bibanon/tubeup/issues/222#issuecomment-1166062367, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKNKRXBIO7YCW737OOQNRNLVQY7N7ANCNFSM5ZZHU35A . You are receiving this because you were mentioned.Message ID: @.***>

upintheairsheep commented 2 years ago

Btw I always put —get-comments on tuneup for some reason. It’s OCD. I already told them and someone found some API URLs that can be put in the WM.

On Fri, Jun 24, 2022 at 4:23 PM Paul Henning @.***> wrote:

I'm opposed to adding in unreliable downloaders to tubeup to make one single user be able to rip videos. Youtube-dl are slow to implement fixes, yt-dlp is better in every single way. YTDL If you want yt-dlp to drop the crypto requirement then ask them to change. we've now added flags and behavior for ourselves and others that is incompatible with YTDL. We'd have a schizophenic setup where most flags work but some don't.

Everyone wants us to change, to adapt to their extremely specific needs. At some point a foot needs to be put down.

@upintheairsheep https://github.com/upintheairsheep it's time for you to buy a cheap linux VPS with enough storage (at least 50GB), that are $6 a month. Do your rips there not on a iphone. And if you want comments from TikTok to be extracted you need to talk to the yt-dlp people.

— Reply to this email directly, view it on GitHub https://github.com/bibanon/tubeup/issues/222#issuecomment-1166062367, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKNKRXBIO7YCW737OOQNRNLVQY7N7ANCNFSM5ZZHU35A . You are receiving this because you were mentioned.Message ID: @.***>

brandongalbraith commented 2 years ago

@upintheairsheep Email me at the email in my bio and we'll figure something out.

vxbinaca commented 2 years ago

Btw I always put —get-comments on tuneup for some reason. It’s OCD. I already told them and someone found some API URLs that can be put in the WM.

YT-DLPs TikTok extractor does not support comments. Talk to them. My answer on this will not change. Ask YT-DLP to add comments scraping. you need to talk to them, once it's supported on their end it will automatically be pulled on our end with --get-comments.

OK. I’ll just use an older version. I’m broke in reality and don’t have a credit card yet.

I can't add schizophrenia to the code because you aren't on the financial network or can't use a VPS host that takes bitcoin. Not happening.

upintheairsheep commented 2 years ago

Again if any of you think I'm being unfair, Twitch extractor was broken for 6 months on YTDL and their maintainers refused to merge the code despite it passing checks and it was tested by others in the community. I had 3 months where I couldn't do any rips of Twitch for months because their negligent staff weren't pushing up fixes or releasing new versions. Absolutely not ever going back to YTDL.

I prize stability over one persons unique situation.

It's time to get a VPS upintheair. They're cheap and with enough disk space can do most channels out there. Bite the bullet and get a VPS it's strongly recommended you not do channel rips from your desktop.

Got it. Have 2 options for iOS, one is yt-archive for archiving channels and videos with comments and metadata, and other is a-shell type pip install tubeup==0.0.26 for twitter and reddit videos and audio draft and YouTube videos with 0 comments.

vxbinaca commented 2 years ago

You could also get a VPS and have the latest version of everything, you just have to let go of your need to rip from a iphone or spend $6 on a real linux environment for the iphone.

vxbinaca commented 2 years ago

Story straight here: You want us to add schizophrenic downloaders because you somehow bought a iPhone, which are too rich for my blood as phones, but you can't spend $6 for a quality iPhone linux app or rent a VPS?

Sound like a priorities problem that isn't mine.

vxbinaca commented 2 years ago

Per Pukaden in Discord:

He can install yt-dlp without the dependences hanging him up:

pip install yt-dlp --no-deps

Re-locking.