search

snarfed / huffduff-video

📺 Extract the audio from videos on YouTube, Vimeo, and other sites and send it to Huffduffer.
https://huffduff-video.snarfed.org/
95 stars 6 forks source link

YouTube Returning Bot Error #52

Open FlocktimusPrime opened 3 months ago

FlocktimusPrime commented 3 months ago

I've started getting an error on YouTube videos:

ERROR: [youtube] BpkLhRubZkw: Sign in to confirm you're not a bot. This helps protect our community. Learn more

snarfed commented 3 months ago

Ruh roh. Sounds like YouTube may have blocked our IP. Lots of details in https://github.com/yt-dlp/yt-dlp/issues/10128 , but I don't see an easy solution there yet.

snarfed commented 2 months ago

Interestingly https://podqueue.fm/ , a much better alternative to huffduff-video, is still happily downloading from YouTube. If this IP block lasts much longer, I may start pointing people there instead.

snarfed commented 2 months ago

Download bandwidth is way down for August, almost certainly due to this, we're on track to do less than half of July's.

image
queuebit commented 2 months ago

My js bookmarklet now just redirects directly to empty huffduffer form, rather than the error I was seeing before. Is that expected behavior (in this scenario), while we wait to see how this plays out?

snarfed commented 2 months ago

@queuebit hmm! No, not expected. Is this bookmarklet on huffduffer.com? Or huffduff-video.snarfed.org?

queuebit commented 2 months ago

Yeah, I think that is it.

Sorry, I should have tested again, first. I just got the error last week and then didn't this week.

eliasgomezsainz commented 2 months ago

For me, the bookmarklet redirects to https://huffduffer.com/add?popup=true&page=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D77Rm8QXpfEw

cameronj86 commented 1 month ago

I've been itching for YouTube audio for the past month. Is anyone aware of any self-hosted alternatives to circumvent the IP blocking? Leaning towards podpodge (will require extra steps) or this (one-to-one replacement but Russian support) if Google doesn't stop picking on huffduffer.

eliasgomezsainz commented 1 month ago

@cameronj86 I am researching alternatives and the one that I like more is PodQueue: https://podqueue.fm/r/76FH6D

FlocktimusPrime commented 1 month ago

I've been itching for YouTube audio for the past month. Is anyone aware of any self-hosted alternatives to circumvent the IP blocking? Leaning towards podpodge (will require extra steps) or this (one-to-one replacement but Russian support) if Google doesn't stop picking on huffduffer.

I just installed yt-dlp on my laptop and figured out how to use it in the command line. Then I use QuickShare to send the resulting mp3s to my phone. I move them to a folder that my podcast app uses as a virtual podcast. Before I settled on that solution, I installed a terminal app on my phone and installed yt-dlp on the phone. That saved me the file transfer, but the transcoding performance was much slower.

cameronj86 commented 1 month ago

I just installed yt-dlp on my laptop and figured out how to use it in the command line. Then I use QuickShare to send the resulting mp3s to my phone. I move them to a folder that my podcast app uses as a virtual podcast. Before I settled on that solution, I installed a terminal app on my phone and installed yt-dlp on the phone. That saved me the file transfer, but the transcoding performance was much slower.

Nice stuff using yt-dlp. My current solution will likely use it as well. It's a cat and mouse game w/ them versus Google, and it will breaks a few times a year, so definitely keep it up-to-date as much as possible (ie: use backports or a trustworthy repo for Debian-based distros to get the latest releases)

Here's my potential setup for those self-hosting:

  1. (One-time) Set podpodge to ingest all files that hit a specified directory.
  2. (One-time) Set up podpodge and create an RSS feed for my podcast app a la huffduff. This feed will be updated whenever a new file hits the folder
  3. (Every time) Convert video.mp4 to audio.mp3 via yt-dlp
  4. (Every time) Trigger script/alias that moves the audio file to a podpodge folder on my server

Here's the blurb on podpodge that seems promising:

Podpodge is a server + client for converting YouTube playlists (or plain audio files in a directory) into audio-only RSS feeds that podcast apps can consume.

BeKwik commented 1 month ago

Here's my YouTube workaround:

  1. Insert "pi" after "youtube" in the URL bar. This will take you to Y2Mate and automatically load the video.
  2. Choose Audio and Convert. I go with 256kbps quality.
  3. When it's done converting, right-click the Download button and Copy Link. (No need to download the file.)
  4. Paste the link at https://huffduff-video.snarfed.org
  5. Go back to YouTube and copy/paste the title and description.

It's pretty simple and quick. Almost as fast as the bookmarklet was directly from YouTube.

snarfed commented 1 month ago

Usage is way down. huffduff-video is dead, long live...PodQueue and self hosting, I guess.

image

snarfed commented 1 month ago

OK all. I don't plan to fight the bot detection arms race, so I think the sad conclusion here is that I turn down huffduff-video, point people to PodQueue, and let others who want to self-host do that.

I'll plan to do that soon. It's had a good almost-10-year run (5109b31f42338c0f0228b7aaaa7fd0d50d1e81f2). So long, and thanks for all the fish, I guess!

FlocktimusPrime commented 1 month ago

It still works on other sites. Are you shutting it down completely?

snarfed commented 1 month ago

Maybe! You're right, it still works with other sites, but looking at the numbers, all other sites comprise maybe 5-10% of current usage. And PodQueue supports those other sites too!

bcomnes commented 1 month ago

FWIW, I shipped a new unauthenticated/scrape based workaround in https://breadcrum.net

It's a service that works like a cross between huffduffer video and pinboard and supports video podcasts. It's private rss feeds only, so it won't work for passive sharing to huffduffer but it will get you personal feed. Still early access but registration is open for now.

No telling how long this will last but I have some other strategies lined up if need be. Also this strategy appears to be a bit more obscure so maybe it will fly under the radar for a while.

dmitrym0 commented 1 month ago

Oh boy.. I've been using huffduff-video since forever. Thanks for all your work on this @snarfed. This feels like another google reader moment, with the open web slowly disappearing right before my eyes.

PodQueue looks like a real nice alternative, but another $5/month subscription is not in my budget. Is this bot block at the IP level? Are you using yt-dl internally?

bcomnes commented 1 month ago

The bot blocking is a black box so it's hard to know exactly what's going on. Blocking can occur at the IP and authenticated account level. IP blocks appear to target ip ranges in hosted environments more aggressively than residential isp ips. YT-dlp, and its underlying mechanisms appear to be the current target.