[Bug] "This helps protect our community." - "Please sign in to confirm you’re not a bot"

[ggtylerr]

Update 21/09/2024: https://github.com/iv-org/invidious/issues/4734#issuecomment-2365205990

---

EDIT by @unixfox: The Invidious team is aware of this issue. It appears that it affects all the software using YouTube. Please refrain from commenting if you have nothing new to bring up. Thank you.

---

Describe the bug This seems to be another block by YouTube - happens on any video, regardless of settings or instance.

Steps to Reproduce

1. Go to any video on Invidious

Logs No logs on browser. In docker logs:

invidious-invidious-refresh-1  | 2024-06-06 22:35:23 UTC [error] get_video: PrMRuA3pd7g : This helps protect our community. Learn more
invidious-invidious-refresh-1  | 2024-06-06 22:35:23 UTC [warn] i18n: Missing translation key "This helps protect our community. Learn more"

Screenshots

Additional context This seems to be a global update, done before 22:23 UTC. (10:23 PM.) From my brief testing, this is present throughout all instances, and regardless of IPv6 address.

EDIT: It looks like this is just rolling out throughout all servers, so some may not be affected yet.

[RustedTerrier]

This doesn't happen to all instances; https://inv.tux.pizza/ still plays videos.

[ggtylerr]

Interesting - I tried mine, yewtu.be, and puffyan, all of which seem to be blocked. Perhaps it's just targeting the more popular instances then?

[Laurendus]

Cannot confirm this. My private instance with two users is affected as well.

[NotARealPersonHere]

i am also getting same thing on a private instance hosted on vultr

edit it was working earlier i updated with docker pull then stopped working

chould just be timing

[ggtylerr]

Okay, it's clear this isn't global yet and it's a standard YouTube rollout. Please refrain from posting "i'm affected/not affected" comments :)

[accessiblepixel]

On my own selfhosted (local) instance, I'm getting both "This helps protect our community." and also a "You may have found a bug" - some of the public interfaces are affected too...

Here's the output (if it's any use) -

Title: Missing hash key: "videoDetails" (KeyError) Date: 2024-06-07T04:39:15Z Route: /watch?v=EXxMCm941WA Version: 2024.04.27-eda7444 @ master

Backtrace

``` Missing hash key: "videoDetails" (KeyError) from /usr/share/crystal/src/hash.cr:1080:9 in 'dig' from /usr/share/crystal/src/json/any.cr:330:3 in 'try_fetch_streaming_data' from src/invidious/videos/parser.cr:111:27 in 'extract_video_info' from src/invidious/videos.cr:395:10 in 'fetch_video' from src/invidious/videos.cr:383:13 in 'get_video:region' from src/invidious/routes/watch.cr:55:15 in 'handle' from lib/kemal/src/kemal/route.cr:13:9 in '->' from src/invidious/helpers/handlers.cr:30:37 in 'call' from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call' from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call_next' from lib/kemal/src/kemal/filter_handler.cr:21:7 in 'call' from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call' from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call_next' from src/invidious/helpers/handlers.cr:94:12 in 'call' from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call' from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call' from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call_next' from src/ext/kemal_static_file_handler.cr:106:14 in 'call' from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call' from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call' from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call' from /usr/share/crystal/src/http/server/request_processor.cr:51:11 in 'handle_client' from /usr/share/crystal/src/fiber.cr:146:11 in 'run' from ??? ```

Edit: Also effects yt-dlp - seems that the "This helps protect our community" is a truncation of the full error message "Sign in to confirm you're not a bot. This helps protect our community. Learn more.", given that usually yt-dlp have multiple ways to download it might not be such an easy workaround:

Log:

[youtube] Extracting URL: https://youtube.com/watch?v=IHifwwHVdUw
[youtube] IHifwwHVdUw: Downloading webpage
[youtube] IHifwwHVdUw: Downloading ios player API JSON
ERROR: [youtube] IHifwwHVdUw: Sign in to confirm you’re not a bot. This helps protect our community. Learn more

[Sommerwiesel]

They are probably A/B testing this, I just (programmatically) did 200 requests on my instance and roughly 1/3 of them returned this bug, the other times, the video loaded just fine. This will most likely get worse the next days, so brace yourself for another downtime of all instances.

[WizardOfWor1969]

They might be geolocating as well. I switched my VPN from US (Silicon Valley) to central america, and all the errors go away.

[ggtylerr]

They might be geolocating as well. I switched my VPN from US (Silicon Valley) to central america, and all the errors go away.

99% sure it's just correlation - A/B tests are gradual and (in this case at least) not specific to regions.

[ggtylerr]

Update: Cobalt seems to have fixed this by implementing a token system: https://github.com/imputnet/cobalt/issues/551

I don't recommend this since it'd be easy to overload (ex: nitter) but it is an option, would be good for temporary use.

[m0istn00dl]

This isn't just happening on invidious instances and third party apps, I've been getting this message trying to watch videos directly off of YTs website from the DDG app on my phone.

[that404nerd]

Lot of instances are not working. It could be youtube doing the 3rd party app blocking

[that404nerd]

This isn't just happening on invidious instances and third party apps, I've been getting this message trying to watch videos directly off of YTs website from the DDG app on my phone.

Works just fine for me. What could be the reason?

[00-kat]

Works just fine for me. What could be the reason?

Similarly, some instances work just fine. As mentioned above, this is probably because YT's A/B testing.

Oh or maybe @m0istn00dl got their IP banned previously by using Invidious or something so youtube.com won't work until they get unbanned.

[that404nerd]

Works just fine for me. What could be the reason?

Similarly, some instances work just fine. As mentioned above, this is probably because YT's A/B testing.

Oh or maybe @m0istn00dl got their IP banned previously by using Invidious or something so youtube.com won't work until they get unbanned.

That sucks tbh, YouTube is really trying to destroy third party apps like invidious but not ban scammers and crypto bots. They don't care and don't value their customers.

[that404nerd]

I am pretty sure that's why revanced yt doesn't work anymore for me.

[ggtylerr]

Oh or maybe @m0istn00dl got their IP banned previously by using Invidious or something so youtube.com won't work until they get unbanned.

Nope. It's pretty much confirmed at this point that basically every client is being blocked and requiring a sign in. Including YouTube's own clients. The only exception found thus far seems to be the TV client (although I could be wrong on this.)

Why exactly this is the case is something only Google can answer - whether it's meant to directly hinder frontends / third-party clients, to prevent unknown (i.e. hard to track) users that don't sign in, or if it's a test to see how much the userbase would be affected by the change. No real way to find out for sure until Google makes some other moves.

(Oh, and also, Google has no way of telling whether an IP visited Invidious, unless you accessed it from their search engine or something. That IP ban claim is ridiculous.)

[m0istn00dl]

Works just fine for me. What could be the reason?

Similarly, some instances work just fine. As mentioned above, this is probably because YT's A/B testing.

Oh or maybe @m0istn00dl got their IP banned previously by using Invidious or something so youtube.com won't work until they get unbanned.

I use PIA VPN on both my phone and my computer, and use streaming optimized servers 90% of the time. If IP bans were on the table, then there wouldn't be any variance between my computer and phone usage.

More likely it's just some tracking javascript or something similar that doesn't happen to play nice with mobile, rich media embeds, and third party apps.

[that404nerd]

That's weird

[that404nerd]

Well seems like it, it might not really ip bans since stuff doesn't work even with vpn.

[unixfox]

@that404nerd that is not a chat discussion. I said to refrain from writing some comments if you have nothing new to bring up.

If you or anyone else want to discuss freely about the issue then join our matrix (https://matrix.to/#/#invidious:matrix.org) or IRC (https://web.libera.chat/?channel=#invidious)

[ic-scm]

I am sorry in advance if this is an unwanted comment in this pinned issue. I know this feature/change might be complicated to implement in a short timespan, especially if the true solution is going to be found earlier, but I noticed in @m0istn00dl's screenshot that the official youtube web app only blocks the video player but not the rest of the video page.

My suggestion is, that maybe, a change could be made to not error out the entire watch page on Invidious when it encounters this error, but to only disable the video player and show the actual video page with all the video information? I may be wrong but I think it's a pretty common solution for people to use an Invidious instance for subscription feeds and viewing video info (description, stats, comments), but use mpv with youtube-dl/yt-dlp to actually watch the videos which still seems to work without any issues, has usually been more reliable, and takes the biggest load off of Invidious instances which use a proxy.

Again, I don't know this so I am sorry if this is too hard or impossible to implement, but I think it would solve a lot of problems for a significant chunk of users of Invidious.

[unixfox]

@ic-scm please create a new github issue for this feature request.

[Brent-Sanchez]

Oh or maybe @m0istn00dl got their IP banned previously by using Invidious or something so youtube.com won't work until they get unbanned.

Nope. It's pretty much confirmed at this point that basically every client is being blocked and requiring a sign in. Including YouTube's own clients. The only exception found thus far seems to be the TV client (although I could be wrong on this.)

Why exactly this is the case is something only Google can answer - whether it's meant to directly hinder frontends / third-party clients, to prevent unknown (i.e. hard to track) users that don't sign in, or if it's a test to see how much the userbase would be affected by the change. No real way to find out for sure until Google makes some other moves.

(Oh, and also, Google has no way of telling whether an IP visited Invidious, unless you accessed it from their search engine or something. That IP ban claim is ridiculous.)

I use your front end pretty regularly, so I'd love to help ya out here. What I have is just circumstancial, since I wasn't setting up to target invidious for a test or anything. I was a little confused'ded over here because I had never seen this error, but I'm a boss at "finding a bug in invidous", just not recently lol. I reset my video card to defualts without a system restart, and immediately got errors thrown all across invidious as well as from my "Open in VLC™" browser extension. If I keep reproducing this error now that I've re-launched the browser (brave-nightly-current realase) and the system, I'll post back to the thread. I can't stress this enough though, this is ALWAYS the case, when I try to utilize my Intel GC and Nvidia card, as apposed to trying to dogknot everything through just the Nvidia card. Intel authors up most of the "don't steal mah sh*t" products that giggles, etc use. It's like quick sand.

[ggtylerr]

I can barely understand half of what that paragraph said. From what I can understand - it's sounding to me like you're just pretty lucky in terms of accessing Invidious. The error is still very much there for everyone else.

[dragonchaser]

https://github.com/TeamNewPipe/NewPipe is still working, maybe you can get some inside from them?

[00-kat]

NewPipe is still working

You sure?

[dragonchaser]

@00-kat at least it is on my phone, while iv is not :) I have the recent version and I do not have seen this error so far...

[dragonchaser]

yt-dl is working randomly.....

[ggtylerr]

Again. It's an A/B test. Some people/clients are affected, some aren't. Like both me and unix said, please don't fill the issue with comments saying "I'm not affected on X" or "Y works but Z doesn't". Just because you aren't experiencing the block yet does not mean it doesn't exist.

[dragonchaser]

@ggtylerr that's not the point. 99% of all iv instances I tested this morning were affected. So I would make the assumption that there is a more complex mechanism in place than just checking the request client. Should have added that to the previous comment sorry :)

[unixfox]

I'm locking this GitHub issue because people can't follow the rules, stating that we should keep this issue clean from unnecessary comments.

If you want to react to this issue and bring up new things, please join our Matrix room or IRC. And if you have useful knowledge to share in this issue, you can create a separate GitHub issue and link it to this one.

[Brent-Sanchez]

I can barely understand half of what that paragraph said. From what I can understand - it's sounding to me like you're just pretty lucky in terms of accessing Invidious. The error is still very much there for everyone else.

"Luck" Come on Tyler mah boy, I'm not THAT Irish, I'm just another pirate huntin for mah booty, yaaarrrr. If you believe the account login you're (not using?) to host your issue is the root of this issue, why not just use "ye ole pivotf00t™" account switcher? 🅷🆃🆃🅿🆂://🆆🆆🆆.🆈🅾🆄🆃🆄🅱🅴.🅲🅾🅼/🅶🅴🆃🅰🅲🅲🅾🆄🅽🆃🆂🆆🅸🆃🅲🅷🅴🆁🅴🅽🅳🅿🅾🅸🅽🆃

I also reccomend "hooked on phonics". I hope this has been relavent and helpful.

[shaedrich]

long-ish quote by @zander678

Honestly, at this point we all know why youtube is doing this, it's because of adds, and invidious' lack of them. Without adds, youtube(and google) don't make as much money, and it's always profit over everything else. So their blocking invidious in an attempt to heard people back to youtube so that they can get their sweet sweet add-cents. And then force them to buy youtube premium, when we could just watch our videos add-free without charge. Google is a greedy corporation, and they will do absolutely anything to stop people from using websites that hinder their money making, even if said website actually let's people watch their videos uninterupted. If youtube wants people to stop using alternate front ends, they should implement a built in add-blocker, and find some other way to make money, but we all know that's not going to happen.

— @zander678

If fixing it is not possible, can the error message at least be reworded to reflect the truth and not YouTube's take on it?

[ggtylerr]

Minimized comment

> "Luck" Come on Tyler mah boy, I'm not THAT Irish, I'm just another pirate huntin for mah booty, yaaarrrr. If you believe the account login you're (not using?) to host your issue is the root of this issue, why not just use "ye ole pivotf00t™" account switcher? 🅷🆃🆃🅿🆂://🆆🆆🆆.🆈🅾🆄🆃🆄🅱🅴.🅲🅾🅼/🅶🅴🆃🅰🅲🅲🅾🆄🅽🆃🆂🆆🅸🆃🅲🅷🅴🆁🅴🅽🅳🅿🅾🅸🅽🆃 > > I also reccomend "hooked on phonics". I hope this has been relavent and helpful.

Each Google account requires a phone number, and burning through accounts is not a good or reliable solution. The current fix being drafted by @unixfox and @SamantazFox (which I won't go over the details of since I know for a fact Google is keeping an eye on this) is a much better solution - just have some patience.

[Raposa-Coltran]

this feels like it needs to be something invidious displays properly, this error should reroute to a "youtube is blocking instances" message or something, especially when the "learn more" just is dead text, atleast untill the problem is fixed properly (tho i imagine it would be good practice to reroute almost any kind of novel error type to such an error in future too)

[unixfox]

this feels like it needs to be something invidious displays properly, this error should reroute to a "youtube is blocking instances" message or something, especially when the "learn more" just is dead text, atleast untill the problem is fixed properly (tho i imagine it would be good practice to reroute almost any kind of novel error type to such an error in future too)

when it's finally implemented, there will be a message explaining what to do in order to solve the issue.

[Raposa-Coltran]

this feels like it needs to be something invidious displays properly, this error should reroute to a "youtube is blocking instances" message or something, especially when the "learn more" just is dead text, atleast untill the problem is fixed properly (tho i imagine it would be good practice to reroute almost any kind of novel error type to such an error in future too)

when it's finally implemented, there will be a message explaining what to do in order to solve the issue.

it feels to me like errors should be updated when the problem is known, not just when theyre fixed, as this leaves users rather confused, before looking it up i assumed my instance had blocked the video itself given the error message, good to know it will be fixed tho

[shaedrich]

it feels to me like errors should be updated when the problem is known, not just when theyre fixed

Especially since (when will this be finally fixed?) this is going on for quite an uncomfortable while and who knows how many people we lost due to the error itself and because they couldn't make heads and tails of the error message and just deemed invidious too buggy, given that not everybody is tech-savvy.

[ggtylerr]

Especially since (when will this be finally fixed?) this is going on for quite an uncomfortable while and who knows how many people we lost due to the error itself and because they couldn't make heads and tails of the error message and just deemed invidious too buggy, given that not everybody is tech-savvy.

I know you're not specifically referring to how long the fix is coming, but I personally know a lot of people are (talking about you, ppl commenting on my site asking when invidious will be fixed >:c ) The patch I was referring to before is fairly complicated, involving multiple stages, decryption, etc. If you know Crystal (which unfortunately I don't), you're more than welcome to help.

[shaedrich]

If you know Crystal (which unfortunately I don't)

Me neither, unfortunately.

when will this be finally fixed? […] The patch I was referring to before is fairly complicated

I didn't want to complain, I'm aware that this takes time. If there's anyone, we should be angry about, then it is Google, not people working hard on a solution. We should be united in this. This is why I think, waiting for a solution will endanger the unity of the community the longer this takes. That's why I think, the rewording of the error message should be done in parallel and as fast as possible so that people know this is not Invidious' fault and YouTube's error message is nothing but euphemisms.

[accessiblepixel]

The 'Learn more' text is a link directly from the YouTube error (which links to YouTube's help pages), it's not one that invidious has added. The difference is that some of it gets trimmed and the link gets made non-clickable.... but I understand that it's a little confusing.

If I remember correctly the full text is "Sign in to prove you're not a bot. This helps protect our community. [Learn More]", so I echo @Raposa-Coltran that it should better reflect what's happening and even if there's no solution maybe there's a better way to handle this and any future issues, than just waiting until there's a fix?

I am looking forward to what Invidious's solution to the issue is (much love to the devs working on this, it's much appreciated what you do <3), although I personally found that using different IP endpoints got my local instance back to being useable again (through a public ipv4 VPN endpoint) as well as yt-dlp, so I'd imagine the IPv6 rotator would also help with this too (for those that have it on a VPS/hosting - although I don't host mine publicly or use ipv6).

[stas2k]

I am looking forward to what Invidious's solution to the issue is (much love to the devs working on this, it's much appreciated what you do <3), although I personally found that using different IP endpoints got my local instance back to being useable again (through a public ipv4 VPN endpoint) as well as yt-dlp, so I'd imagine the IPv6 rotator would also help with this too (for those that have it on a VPS/hosting - although I don't host mine publicly or use ipv6).

In my case, IPv6 on Hetzner Cloud did not do anything, the error still persists.

[barelylit]

Are people still having this issue? It started working again for me a few days later, i presumed it had been fixed. Its been fine since.

[shaedrich]

@barelylit I even had this issue on https://invidious.privacyredirect.com a couple of days ago, even though it usually works fine unlike a bunch of other instances.

name	is blocked by YouTube	version	location	health	cors	api
invidious.fdn.fr	✔	2024.07.28-3649d2e3	🇫🇷 FR	99.677	✔	✔
inv.tux.pizza	✔	2024.04.27-eda7444	🇺🇸 US	99.343	✔	✔
invidious.flokinet.to	✔	2024.05.27-1ae14cc2	🇷🇴 RO	98.514	✔	✔
invidious.privacydev.net	✔	2024.05.27-1ae14cc2	🇫🇷 FR	97.824	✔	✔
invidious.protokolla.fi	❌	2024.04.27-eda7444	🇫🇮 FI	100.0	✔	✔
invidious.nerdvpn.de	✔	2024.07.27-90e94d4e	🇺🇦 UA	53.636	✔	✔
invidious.private.coffee	✔	2024.04.27-eda7444c	🇦🇹 AT	99.877	✔	✔
invidious.jing.rocks	❌	2024.07.27-90e94d4e	🇯🇵 JP	99.582	✔	✔
invidious.perennialte.ch	❌	2024.07.27-90e94d4	🇦🇺 AU	100.0	✔	✔
yt.artemislena.eu	✔	2024.04.27-eda7444	🇩🇪 DE	99.967	✔	✔
invidious.materialio.us	1	2024.04.27-eda7444	🇳🇿 NZ	99.532	❌	❌
invidious.reallyaweso.me	✔	2024.07.21-325561e	🇩🇪 DE	99.214	❌	✔
inv.in.projectsegfau.lt	✔	2024.04.27-eda7444	🇮🇳 IN	84.48	✔	✔
iv.datura.network	✔	2024.04.27-eda7444	🇫🇮 FI	99.738	✔	✔
invidious.darknessservices	❌	2024.04.27-eda7444	🇺🇸 US	92.819	✔	✔
yt.drgnz.club	2	2024.07.27-90e94d4e	🇨🇿 CZ	99.126	✔	✔
yewtu.be	3	2024.08.05-7e4f0de	🇩🇪 DE	99.911	❌	❌
invidious.incogniweb.net	✔	2024.07.27-90e94d4e	🇺🇸 US	99.293	✔	✔
iv.melmac.space	❌	2024.07.27-90e94d4e	🇩🇪 DE	98.907	✔	✔
inv.nadeko.net	4	-	🇨🇱 CL	89.552	-	-
invidious.drgns.space	5	-	🇺🇸 US	5.549	-	-
invidious.privacyredirect.com	✔	2024.04.27-eda7444	🇫🇮 FI	99.872	✔	✔

¹ Could not check out a connection in 5.0 seconds (DB::PoolTimeout) ² 504 Gateway Time-out ³ The media could not be loaded, either because the server or network failed or because the format is not supported. ⁴ Doesn't load at all ⁵ Redirects to redirect.invidious.io

[unixfox]

Everyone in the Invidious team does not have unlimited free time, we work at our own pace. Like @ggtylerr explained, it's much more complicated to solve than any previous YouTube breakage. Hence, why it takes a lot of long time to get a fix.

It's work in progress. For the more curious, here are the PR for fixing the issue: https://github.com/iv-org/invidious/pull/4772 and https://github.com/iv-org/invidious/pull/4789

We have no plan to alter the error message until a solution has been implemented. We know we are probably loosing many users, but it's part of the life of an open source project. Usually an open source project is run by volunteers and users should not expect the same reliability as a product run by a company where many developers are working on the product every day.

[mas1701]

@unixfox As my instance is also affected after running very smoothly for quite a long time, I want to express my thanks for all the work you and other devs have been putting into Invidious and are still continuing to do so, greatly appreciated! Just take the time it needs. No pressure.

[hemna]

fwiw, my instance got hit by this as well.

2024-08-09 13:53:37 UTC [debug] YoutubeAPI: Using endpoint: "/youtubei/v1/player"
2024-08-09 13:53:37 UTC [error] get_video: t4_Q2re0Hw4 : This helps protect our community. Learn more
2024-08-09 13:53:37 UTC [warn] i18n: Missing translation key "This helps protect our community. Learn more"
2024-08-09 13:53:37 UTC [info] 500 GET /watch?v=t4_Q2re0Hw4 53.88ms
2024-08-09 13:53:38 UTC [info] 200 GET /api/v1/auth/subscriptions 1.55ms

I'm running from a container image

services:

  invidious:
    image: quay.io/invidious/invidious:latest-arm64 # ARM64/AArch64 devices
...

[Feodor2]

I suggest to disables automatic redirect at once, or you will burn all working instances left.

[unixfox]

I'm locking again this issue because some people can't behave sorry.

There will be an announcement when a fix is deployed. It's work in progress, we have solutions, Invidious is not halted.