jagrosh / MusicBot

🎶 A Discord music bot that's easy to set up and run yourself!
https://jmusicbot.com
Apache License 2.0
5.39k stars 2.58k forks source link

"Sign in to confirm you’re not a bot." error with YouTube videos #1588

Open MichailiK opened 5 months ago

MichailiK commented 5 months ago

Bug Description

Playing YouTube videos may fail with the error "Sign in to confirm you’re not a bot."

YouTube seems to have recently rolled out this measure. It's affecting not only third party players/clients, but also YouTube's official website & apps when logged out. Several communities have come across a few discoveries. Here's a summary of what I've seen:

  1. This is an IP block.
  2. Instead of just banning one IP, it looks YouTube is blocking whole ASNs/IP ranges. If you encounter this issue, it's likely a "neighboring" IP address was bombarding YouTube, causing them to block the ASN/IP range you're in.
  3. The cobalt.tools developers seem to have found a workaround using OAuth2. This currently has a PR in the lavaplayer youtube source manager, however it seems a bit risky.

We are in the process of implementing OAuth2 login for JMusicBot in #1670. Use this with an alternative/burner!!! Google account to play tracks again on blocked IPs.

Checklist

PouletteMC commented 5 months ago

Also having this issue, if anyone has a solution or a quick and dirty workaround, I'd be down

TheBjoel2 commented 5 months ago

downloading youtube videos through tor might be a workaround for this problem. just tested this, it works

$ torsocks yt-dlp https://youtu.be/AL5vANu2gok                                                                                                                                                               
[youtube] Extracting URL: https://youtu.be/AL5vANu2gok
[youtube] AL5vANu2gok: Downloading webpage
[youtube] AL5vANu2gok: Downloading ios player API JSON
[youtube] AL5vANu2gok: Downloading player 4fc7f9fa
[youtube] AL5vANu2gok: Downloading m3u8 information
[info] AL5vANu2gok: Downloading 1 format(s): 302+251
[download] Destination: Black Holes Explained For Gen-Z [AL5vANu2gok].f302.webm
[download] 100% of   48.22MiB in 00:03:40 at 223.78KiB/s
[download] Destination: Black Holes Explained For Gen-Z [AL5vANu2gok].f251.webm
[download] 100% of   10.01MiB in 00:00:49 at 207.41KiB/s
[Merger] Merging formats into "Black Holes Explained For Gen-Z [AL5vANu2gok].webm"
Deleting original file Black Holes Explained For Gen-Z [AL5vANu2gok].f302.webm (pass -k to keep)
Deleting original file Black Holes Explained For Gen-Z [AL5vANu2gok].f251.webm (pass -k to keep)
cmorley191 commented 5 months ago

This other repo fixed the issue by adding support for Oauth2, essentially allowing people to create dummy accounts for the bots to log in with: https://github.com/imputnet/cobalt/issues/551

Is this something we'd have to request support for in lavaplayer? Or is MusicBot responsible for querying the youtube API?

Ozak93 commented 5 months ago

I'm facing the same issue:

Jun 17 23:49:14 env[441177]: [23:49:14] [ERROR] [AudioHandler]: Track BJhF0L7pfo8 has failed to play Jun 17 23:49:14 env[441177]: com.sedmelluq.discord.lavaplayer.tools.FriendlyException: Sign in to confirm you’re not a bot Jun 17 23:49:14 env[441177]: at dev.lavalink.youtube.clients.skeleton.Client.getPlayabilityStatus(Client.java:72) Jun 17 23:49:14 env[441177]: at dev.lavalink.youtube.clients.skeleton.NonMusicClient.loadTrackInfoFromInnertube(NonMusicClient.java:98) Jun 17 23:49:14 env[441177]: at dev.lavalink.youtube.clients.skeleton.StreamingNonMusicClient.loadFormats(StreamingNonMusicClient.java:35) Jun 17 23:49:14 env[441177]: at dev.lavalink.youtube.track.YoutubeAudioTrack.loadBestFormatWithUrl(YoutubeAudioTrack.java:175)

Moonchild6279 commented 5 months ago

I can confirm that with my VPS Server, it logs a "Sign in to confirm you’re not a bot." error everytime I want to play a youtube video on my server.

HOWEVER, just recently I found out my old instance of my bot in my local machine work perfectly with youtube. Perhaps it's something to do with how the I.P of my local machine is tied to my youtube account perhaps? maybe youtube blocked requests from my server because of that.

Derahex commented 4 months ago

a bot called "Nero Bot" manages to still play YT videos- would be worth looking into how they manage past the whole thing with their "nodes".

TheBjoel2 commented 4 months ago

a bot called "Nero Bot" manages to still play YT videos- would be worth looking into how they manage past the whole thing with their "nodes".

1) Nero Bot is not open source 2) Minerea is also able to play YT. Not open source 3) Chances are they are paying for a YT API or idk

samusaran commented 4 months ago

Some people over lavaplayer suggests to use lavaplayer-fork to solve youtube issues

MichailiK commented 4 months ago

Some people over lavaplayer suggests to use lavaplayer-fork to solve youtube issues

We're already using lavalink's lavaplayer fork, which is still receiving maintenance. For this issue specifically, a simple update is very unlikely to fix anything, because this block also affects all of YouTube's own apps/clients.

samusaran commented 4 months ago

So I guess this is a problem to any bot playing from YouTube as a source, right?

MichailiK commented 4 months ago

See the various other issues on the internet on this topic for a rough history of this. The cobalt.tools developers seem to have found a workaround related to OAuth2, however this has not been implemented in any lavaplayer fork yet, and also seems a bit risky.

samusaran commented 4 months ago

The issue is fixed for me on version 0.4.2 🎉

MichailiK commented 4 months ago

As this is an issue that affects every project using YouTube, several communities have come across a few discoveries. Here's a summary of what I've seen:

  1. This is an IP block.
  2. Instead of just banning one IP, it looks YouTube is blocking whole ASNs/IP ranges. If you encounter this issue, it's likely a "neighboring" IP address was bombarding YouTube, causing them to block the ASN/IP range you're in.
  3. The cobalt.tools developers seem to have found a workaround using OAuth2. This currently is not implemented in lavaplayer has a PR in the lavaplayer youtube source manager, however it seems a bit risky.
  4. IPv6 rotation might also work, which lavaplayer supports & encourages using. We haven't put much thought about supporting IPv6 rotation on JMusicBot yet though.

In my opinion, I see three main ways JMusicBot could move forward:

  1. Wait for OAuth2 support to be arrive in the lavaplayer youtube source manager & make use of that
  2. Allow configuring IPv6 rotation for users on VPS' as they are most commonly affected by these blocks.
  3. Implement support for an external tool (cobalt.tools) as a source, which have circumvented this block. I'm not sure if they would appreciate the additional traffic from JMusicBot users though.

Only my two cents though, no discussion around this has happened yet.

T-h-o-r-p-e commented 4 months ago

This same issue has started occurring for me, has an easy solution been found yet? (one that a novice like me could implement)

MichailiK commented 4 months ago

There is no solution at the moment.

Snaacky commented 4 months ago

This started affecting me on a Hetzner IP range yesterday. Not a solution per se, but managed to temporarily workaround it by routing my jmusicbot Docker container through my WireGuard Docker container running a config from a paid VPN service.

T-h-o-r-p-e commented 4 months ago

This started affecting me on a Hetzner IP range yesterday

I'm also using Hetzner. Not sure if that's a coincidence or not.

MichailiK commented 4 months ago

I'm also using Hetzner. Not sure if that's a coincidence or not.

This is expected. As I mentioned earlier, it seems like YouTube is banning entire ASNs/IP ranges, rather than just individual IP addresses. Chances are pretty high that all of Hetzner’s ASNs/IPs have been banned.

Ruok2bu commented 4 months ago

As a test, on my server computer i created a new google account (so it appears that the registration is occurring from the banned ip) and logged in. I was able to play youtube video's in the browser! But when i log out in browser, once again im presented with the error. So if the bot can log into youtube, that should solve the problem for everyone.

tag27 commented 4 months ago

So if the bot can log into youtube, that should solve the problem for everyone.

No, apparently this block affects both with or without an account, on the same IP. See: https://github.com/lighttube-org/LightTube/issues/154 https://github.com/yt-dlp/yt-dlp/issues/10085

Edit: Oops, I mentioned another problem without even paying much attention. Yes, you can "fix" it by connecting with an account (https://github.com/yt-dlp/yt-dlp/issues/10128#issuecomment-2185261677), but I don't think it would be very ideal for a project like this (Not to mention that Google can easily end up discovering this repository and try to destroy it, I think). The only four solutions I've seen for now are:

While there is no 100% working solution, the only way to continue using the bot is to go to IPs, servers and/or VPNs that YouTube has not (yet) blocked.

mersal-developing commented 4 months ago

this also affects me i used netrc file and add the credentials of youtube but still get the same error

ROBERT-MCDOWELL commented 4 months ago

YT seems to want to shoot a bullet in his own foot.... I have the same issue from 4 servers in 4 different DC. well done YT, soon you are going to ban yourself as some of your servers are hosted in the same DC of ours!

BenCos17 commented 4 months ago

can confirm it's happening for me on hetzner also
not using musicbot though

samusaran commented 4 months ago

It’s also happening to me on Aruba (Italian hosting provider)

xFanexx commented 4 months ago

Same. Not working for me either. Using a Hetzner Dedicated Server with Windows Server 2022 // Docker Desktop.

hydrogenicMetallium commented 4 months ago

i am just popping in to say that after a bunch of tinkering with my OVHcloud server yesterday i only found out that if you run jmusicbot and then a VPN (OpenVPN + Windscribe is what ive used) on top of it then will the bot finally be able to play youtube links without any issues, granted this is still a workaround and you may have to restart your vpn again each time if youtube happens to ban the one the vpn is running on but so far ive had 0 issues since ive went and used this solution. just thought id let everyone know about this since ive seen a bunch of people still trying to figure out how to get their bot to play yt links so yeah i hope this helps

Zawodowiec1532 commented 4 months ago

same on OVH server.. any resolution is planned?

BlobIsBack commented 4 months ago

same on OVH server.. any resolution is planned?

Using cloudflare warp on my racknerd vps fixed the issue BUT i lost accès to ssh and now use vnc because I didn't set up cloudflare warp correctly and I'm too lazy to reinstall it, I reccomend you to search on google stuff like "can't accès ssh after installing cloudflare warp" and you will probably find something to not have the same issue as me

Sorry if my English is bad

C0la149 commented 4 months ago

Hello! I also encountered the same problem. Has anyone found a solution to this problem? My server is from Hentzer. Located in Finland! изображение изображение Of course I have a nextcloud cloud. But maybe someone already knows the solution to this problem?

tag27 commented 4 months ago

But maybe someone already knows the solution to this problem?

No, it doesn't have a 100% effective solution so far. There are repositories with the same problem, and others have some pulls that can fix the problem, but many will fix it temporarily.

The best thing to do is to wait.

ROBERT-MCDOWELL commented 4 months ago

well, why not to totally boycott the corpofascists?

C0la149 commented 4 months ago

But maybe someone already knows the solution to this problem?

No, it doesn't have a 100% effective solution so far. There are repositories with the same problem, and others have some pulls that can fix the problem, but many will fix it temporarily.

The best thing to do is to wait.

Wait what? How long will the ban last? or until a complete solution comes out?

Ruok2bu commented 4 months ago

One thing that can work and its not ideal. But what if the bot downloads the youtube video and then plays the audio through the bot? And optionally caches the downloaded video into a folder structure on the hard drive so it doesnt need to contact youtube servers again. I realize this method can bloat the hard disk with GB's of files, but if they are also converted from mp4 (m4v) to ogg (not mp3 because ogg offers better compression than mp3) then some disk space would be saved.

I realize this method wont unban everyone's IPs but at least it would allow us to get around the ban.

DaniDipp commented 4 months ago

@Ruok2bu There is no difference between downloading and streaming a video from YouTube's point of view. It won't get around the ban.

Ruok2bu commented 4 months ago

@Ruok2bu There is no difference between downloading and streaming a video from YouTube's point of view. It won't get around the ban.

But at least adding that ability could let us schedule when the bot accepts songs to download (in other words it wont trigger the bans all the time). For example, allowing music requests for set time of the day or up to x amount per hour so youtube is less likely to block.

MichailiK commented 4 months ago

This is highly unlikely to fix anything in a VPS/cloud service context (which are the people who get affected by this block the most.) Based on observations made by the community about this block, YouTube is very likely applying a ban to the whole ASN (IP range.) If only one bad actor within your "IP range" is misbehaving by downloading lots of videos in a short period, you will end up being affected through no fault of your own.

Ruok2bu commented 4 months ago

Then there's only one way out of this. Some rich person needs to buy a ton of google stocks, get on the board and then use their veto power to change the rules.

ROBERT-MCDOWELL commented 4 months ago

@MichailiK well, imagine someone using yt-dlp in an airport which usually has an ip range, good luck to ban an entire airport without any consequences!

C0la149 commented 4 months ago

@MichailiK well, imagine someone using yt-dlp in an airport which usually has an ip range, good luck to ban an entire airport without any consequences!

xD

tag27 commented 4 months ago

But maybe someone already knows the solution to this problem?

No, it doesn't have a 100% effective solution so far. There are repositories with the same problem, and others have some pulls that can fix the problem, but many will fix it temporarily. The best thing to do is to wait.

Wait what? How long will the ban last? or until a complete solution comes out?

They also don't know how long this ban lasts, they say it's 72 hours or it's permanent. (https://github.com/yt-dlp/yt-dlp/issues/10128#issuecomment-2180723436)

Zawodowiec1532 commented 4 months ago

The solution might be to just log in. I checked it on my VPS and when i logged in, it started to work. yt-dlp allow to login via parameters: https://github.com/yt-dlp/yt-dlp?tab=readme-ov-file#authentication-options So I think you need to implement option to provide credentials for yt-dlp

trodrigues commented 3 months ago

This started affecting me on a Hetzner IP range yesterday. Not a solution per se, but managed to temporarily workaround it by routing my jmusicbot Docker container through my WireGuard Docker container running a config from a paid VPN service.

@Snaacky would you happen to have a bit of guidance on you set up jmusicbot to go through the VPN? Is it a container level configuration?

Snaacky commented 3 months ago

@trodrigues This is what I'm using:

services:
  jmusicbot:
    container_name: jmusicbot
    image: bas0korver/jmusicbot:latest
    restart: unless-stopped
    network_mode: "container:wireguard"
    volumes:
     - ./config:/opt/jmusicbot/config
services:
  wireguard:
    container_name: wireguard
    image: linuxserver/wireguard
    restart: unless-stopped
    environment:
      - PUID=1000
      - PGID=1000
    cap_add:
      - NET_ADMIN
      - SYS_MODULE
    volumes:
      - ./config:/config
      - /lib/modules:/lib/modules
    privileged: true
    sysctls:
      - net.ipv4.conf.all.src_valid_mark=1
      - net.ipv4.ip_forward=1
      - net.ipv6.conf.all.disable_ipv6=1
cmorley191 commented 3 months ago

The lavaplayer oauth PR we are (maybe) waiting on has been making progress.

@MichailiK You mentioned a couple times that you think the idea of having JMusicBot users logging in with oauth seems "a bit risky." Could you explain that? I don't understand what the risks are.

MichailiK commented 3 months ago

Google could figure out it is not a legitimate client playing back videos. As using JMusicBot technically violates YouTube's TOS, they could terminate your Google account. The OAuth2 PR also states at the end this should not be used on your main Google account.

(For clarity, the OAuth2 API that's being used here is unofficial & undocumented, it's only meant to be used by (Smart) TV makers as far as I'm aware.)

Zawodowiec1532 commented 3 months ago

Google could figure out it is not a legitimate client playing back videos. As using JMusicBot technically violates YouTube's TOS, they could terminate your Google account. The OAuth2 PR also states at the end this should not be used on your main Google account.

(For clarity, the OAuth2 API that's being used here is unofficial & undocumented, it's only meant to be used by (Smart) TV makers as far as I'm aware.)

You can make warning to dont use main account, just create new one :)

Daniyalkhan55 commented 3 months ago

how to fix this bug "Sign in to confirm you’re not a bot" while using the distube-ytdl package

z80sui commented 3 months ago

mark

mwessman commented 3 months ago

So is there any solution or workaround for this yet?

MichailiK commented 3 months ago

The only workaround at the moment is to play music from a non-banned IP address. We're currently waiting for upstream to implement OAuth2 login & then consider adding it to JMusicBot.