Got error: "Sign in to confirm that you're not a bot"

[DuplicantWood]

Official Instance

• [x] The bug is reproducible on the official hosted instance, or is API-related.

Describe the bug

On my private instance and with some others i have tested in the public list i am seeing "Got error: "Sign in to confirm that you're not a bot"" instead of a video

Here is a link from a public instance with the same problem:

https://piped.r4fo.com/watch?v=2G-L0u_L0qU

To Reproduce

Open this url on this instance

https://piped.r4fo.com/watch?v=2G-L0u_L0qU

See error

Expected behavior

Video to play normally

Logs/Errors

org.schabi.newpipe.extractor.exceptions.ContentNotAvailableException: Got error: "Sign in to confirm that you're not a bot" at org.schabi.newpipe.extractor.services.youtube.extractors.YoutubeStreamExtractor.checkPlayabilityStatus(YoutubeStreamExtractor.java:1016) at org.schabi.newpipe.extractor.services.youtube.extractors.YoutubeStreamExtractor.onFetchPage(YoutubeStreamExtractor.java:876) at org.schabi.newpipe.extractor.Extractor.fetchPage(Extractor.java:60) at org.schabi.newpipe.extractor.stream.StreamInfo.getInfo(StreamInfo.java:77) at org.schabi.newpipe.extractor.stream.StreamInfo.getInfo(StreamInfo.java:72) at org.schabi.newpipe.extractor.stream.StreamInfo.getInfo(StreamInfo.java:67) at me.kavin.piped.server.handlers.StreamHandlers.lambda$streamsResponse$0(StreamHandlers.java:54) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317) at java.base/java.lang.VirtualThread.run(VirtualThread.java:309)

Browser, and OS with Version.

Happens on multiple browsers and devices

Additional context

I'm assuming it's related to something google/youtube have done as invidious instances are having similar issues.

https://github.com/iv-org/invidious/issues/4734

[opusforlife2]

Some instances are working, some aren't. But it's probably gonna spread to all of them.

[milutinke]

Is there any solution for this?

[00-kat]

Is there any solution for this?

Cobalt implemented OAuth, but other than that there's not been much in terms of solving this issue...

[milutinke]

Is there any solution for this?

Cobalt implemented OAuth, but other than that there's not been much in terms of solving this issue...

Well, maybe it would be good implementing it as an optional feature, so instances could continue running until there is a better solution.

[Anoerak]

Restarting all containers fixed it for me tonight.

[AndrewHoang1995]

Restarting all containers fixed it for me tonight.

Really !!!

[LivingWithHippos]

Restarting all containers fixed it for me tonight.

Didn't work for me, you probably got "B" tested ;D

jokes aside, I have another request that may warrant a separate issue: I opened a page to use the share button and copy the original youtube link, but the error message "eats" away all the UI, I think it would be nice if a partial interface could still be shown in this case

[Tamarindo94]

This is happening to me on the majority of videos (~8 out of 10) and on different instances, to the point I had to go back to using vanilla youtube. Sadly, I dont know enough about piped or youtube's inner workings to be able to help, but I hope I can bring more attention to this issue. My fear is that this change could make piped and other frontends unusable sooner than we think if a solution isn't found

[alimovlex]

Is there any solution for this?

Yes. Just obtain youtube API keys and paste the API_SECRET to src/main.js function at 127th line.

getAuthToken() {
            return this.getPreferenceString("authToken" + this.hashCode(this.authApiUrl()));
},

Everything works.

[alimovlex]

This is happening to me on the majority of videos (~8 out of 10) and on different instances, to the point I had to go back to using vanilla youtube. Sadly, I dont know enough about piped or youtube's inner workings to be able to help, but I hope I can bring more attention to this issue. My fear is that this change could make piped and other frontends unusable sooner than we think if a solution isn't found

You can host your own instance instead and paste the token as I have written above.

[AndrewHoang1995]

You can host your own instance instead and paste the token as I have written above.

If I use Docker, how can I rebuild it? I am looking forward to an update from the team regarding documentation and Docker for this temporary solution

[clot27]

This is coming so often on official (piped.video) instance, approximately 7 of 10 videos. I haven't tried to change instance yet, I have switched to invidious.

[opusforlife2]

I haven't tried to change instance yet

This is the best workaround though. Works 100% of the time. You rarely have to change it twice.

[danl76]

Your modify the main.js like this?

getAuthToken() { return this.getPreferenceString("authToken" + this.hashCode(API_KEY)); },

Where API_KEY = your yt api key

Also how do get into the docker container to modify main.js? I tried bash but that'd didn't work.

[alimovlex]

Your modify the main.js like this?

getAuthToken() { return this.getPreferenceString("authToken" + this.hashCode(API_KEY)); },

Where API_KEY = your yt api key

Also how do get into the docker container to modify main.js? I tried bash but that'd didn't work.

Unfortunately, no.

getAuthToken() {
            return this.getPreferenceString("YOUR_API_TOKEN_HERE" + this.hashCode(this.authApiUrl()));
},

API_KEY is API_SECRET in your profile.

[alimovlex]

You can host your own instance instead and paste the token as I have written above.

If I use Docker, how can I rebuild it? I am looking forward to an update from the team regarding documentation and Docker for this temporary solution

I am not facile with Docker, unfortunately. Instead, I host my own private instance on a VPS.

[LivingWithHippos]

Your modify the main.js like this?

getAuthToken() { return this.getPreferenceString("authToken" + this.hashCode(API_KEY)); },

Where API_KEY = your yt api key

Also how do get into the docker container to modify main.js? I tried bash but that'd didn't work.

When bash doesn't work try sh ;D (or ash for alpine)

Where is the file located BTW?

edit:

I use these aliases in bash (or remove function for zsh) to quickly find and then enter a container with bash or sh (yeah I should env I know)

# get the container name if you don't remember, example: "dclist piped"
function dclist() {
    if [ -z "$1" ]; then
        docker ps -a --format "table {{.ID}}\t{{.Names}}" | (read -r; printf "%s\n" ""; sort -k 2);
    else
        docker ps -a --format "table {{.ID}}\t{{.Names}}" | (read -r; printf "%s\n" ""; sort -k 2) | grep "$1";
    fi
}

# enter a container with bash or sh, example: "dcsh piped"
function dcbash() { docker exec -it "$1" /bin/bash; }
function dcsh() { docker exec -it "$1" /bin/sh; }

[LivingWithHippos]

Partial docker instructions, untested, try only if you know what this means:

1. open your piped instance
2. right click -> view page source or whatever your browser calls it
3. find out the name of the javascript bundled file, mine is <script type="module" crossorigin src="/assets/index-WHXXydFZ.js"></script>

At this point you can either extract it from the container, change it and mount it back (easier, persist across restarts, may break the frontend if the containers are updated) or manually edit it in place (no time so will do this for now, may need restart of the container, may break anyway or just not work) or add the sed command to the start script (it's already doing something similar for the instance name)

5. enter the piped frontend container with its name, I used docker exec -it "piped-frontend" /bin/sh;
6. enter the assets folder with cd /usr/share/nginx/html/assets
7. swap the authToken with sed -i 's/"authToken"/"YOUR_TOKEN"/g' index-YOUR_FILE.js
8. ???
9. If you force refresh the page and check the sources again you can click on the js file and see if it has been changed or not

[enticedwanderer]

I've tried to do what above was suggested with the substitution of the authToken with an API key for Youtube Data v3 (plain API key not OAuth) and it just results in completely broken piped frontend. No trace or any message in the console. I have confirmed that it is substituted correctly in the browser with cache disabled (just in case).

What's supposed to happen when you swap out the hardcoded authToken string? Can someone provide clearer instructions on how to do this?

I'm also confused what "API_SECRET" referenced above is. Is it not the value of the API key?

[Bnyro]

Everything written above about patching getAuthToken() is not true, that authToken is only used to authenticate at Piped instances with your Piped account, it's not related to the visitor tokens used for extracting data from YouTube.

Obtaining YouTube API keys and only setting them in the frontend doesn't solve the problem at all, because the YouTube API key would need to be used at the Piped backend anyways ...

[Bnyro]

You probably want to save yourself that time and either get a new IP address that's not blocked from YouTube or put a proxy / VPN in front of your instance.

[milutinke]

You probably want to save yourself that time and either get a new IP address that's not blocked from YouTube or put a proxy / VPN in front of your instance.

Invidious made a IPv6 rotation tool, so that is another way of solving the issue. It needs a bit of setup, but works pretty good on a couple of instances I saw using it.

[bonswouar]

Invidious made a IPv6 rotation tool, so that is another way of solving the issue. It needs a bit of setup, but works pretty good on a couple of instances I saw using it.

Unfortunately lots of instances had their whole IPV6 /64 range blocked recently. I guess it depends of the provider

[Bnyro]

Unfortunately lots of instances had their whole IPV6 /64 range blocked recently. I guess it depends of the provider

Yes, all my Hetzner IPv6 ranges are blocked too since some days :/

[milutinke]

I heard https://stylenhost.com/ still works. Maybe we should compile a list of providers that work and don't work.

[bonswouar]

Maybe we should compile a list of providers that work and don't work.

Not sure this is a good idea, could make things much easier for Google :sweat_smile:

(although it doesn't hurt to warn people that hetzner is blocked for example)

[IngwiePhoenix]

Ran into this issue a while ago and haven't had time to look into it. For the meantime I guess I will just deploy gluetun to just use a VPN instead. Because my VPS is indeed with Hetzner... Welp, that's unfortunate. Aside from IPv6 rotation, is there any other idea on how to fix that issue? Thanks!

[danl76]

Being it was the new-pipe extractor that broke, and they just fixed the issue on their side, couldn't we get their updated extractor and plug it in?

[enticedwanderer]

Latest piped-backend seems to have incorporated the change already. However, for me it doesn't change anything, even with the latest docker image I get the antibot prompt. Exact same instance from my home IP address works fine without any changes, so IP reputation also plays a role.

[CelianGdfrd]

Ran into this issue a while ago and haven't had time to look into it. For the meantime I guess I will just deploy gluetun to just use a VPN instead. Because my VPS is indeed with Hetzner... Welp, that's unfortunate. Aside from IPv6 rotation, is there any other idea on how to fix that issue? Thanks!

Hi, IPV6 rotation didn't help for me, as my whole IP range seems to be restricted. Can't deploy a new IPV6 to my server, and ran an update without more luck, so I'm just going to try Gluetun aswell.

[Arcegis]

I can confirm that pretty much all instances, even brand new deployed ones, receive the message "Sign in to confirm you are not a bot." Besides, YouTube now streams unskippable ads, even with UBlock Origin...

[ghost]

This issue is cropping up more and more, Google really is playing whack-a-mole with piped instances, I recommend passing the API through multiple proxies via proxychains and spoofing it to look more like a real user

[IngwiePhoenix]

I did get Gluetun to work but now the backand is snails-slow in returning basically any video or thumbnail. Have any of you had better luck? Provider is ProtonVPN. I doubt they would allow traffic through tor, and I honestly am not sure how to make a transparent proxy out of tor to just route the piped-backend through...

However, the JMusicBot Discord bot that I selfhost - from home - does still work with no bans or alike and I doubt their dependency, LavaPlayer, is doing a whole lot different to piped's backend.

Sidenote, why is it poking into a Matrix channel exactly? I noticed that while peeking at the logs. Found it curious.

[mpeter50]

I doubt they would allow traffic through tor, and I honestly am not sure how to make a transparent proxy out of tor to just route the piped-backend through...

Wouldn't that be even slower?

---

Maybe it would be somewhat of a solution to spin up instances running on or proxied to popular public networks, like coffee shops, schools and universities, libraries, whatever. Of course easiest if you work there and you wont get into trouble for it. Maybe also on residential networks when you know that you can get a new IP address relatively easily (like power cycling the ISP router if you can afford the outages for yourself, or something like that).

Yeah yeah full IPv6 ranges are getting blocked.. of VPS providers. But can they afford this for residential IPv6 ranges? And then the same applies to IPv4 addresses, which might (?) be changeable less often, but which is also even hotter with common usage when you are behind CGNAT.

The point would not be just to try to see if a popular piped instance could run without ban on a network that is basically already accounted for by google as one that generates relatively high traffic. But also to make this shit tactic of google unfeasible to them, by making it break youtube for a lot of common users, so that they would need to consider stopping to do that. And then it might also be beneficial that users who directly use youtube.com on that network would be solving the verifications if they want to watch it.

What do you think? Does this sound too silly, or is it flawed?

[IngwiePhoenix]

@mpeter50

Wouldn't that be even slower?

Set hops to 1 and it wouldn't, actually. More hops, nore rings around the onion(routing). Since this is just Youtube, all we care about is masquerading the outgoing IP; kinda like NAT. So with Tor set to 1 hop, it could actually be fine o.o Once a connection ("circuit") is established, it is pretty good.

Also, we keep talking IPv6 ranges; what about IPv4? I am drawing a blank here, but what if you reduced the Docker CNI to single stack IPv4 and thus made the backend only operate on that - thus using the host's outgoing public IPv4 for requests. Is that viable?

But wait, "public network"... Cloudflare. I have an idea - well, a thought. Their WARP service provided through 1.1.1.1 can also act as a VPN. I'll see if I can get WARP to work inside Docker and replace that with Gluetun and see what happens after that. Who knows...?

[codenyte]

I love the idea.

Check this out: https://github.com/cmj2002/warp-docker

There's also an open feature request for Gluetun to implement Cloudflare WARP support: https://github.com/qdm12/gluetun/issues/1738

[mycodedoesnotcompile2]

@mpeter50 + @IngwiePhoenix, you are right. The future to counter these IP blacklisting is to use different tunneling networks to offload and to "spread" the traffic. Dividing and ruling.

@codenyte, thanks really interesting.

Anyway, what I can tell so far is that, on a cloud environment where I have currently my Invidious instance blocked with this awful "Sign to confirm" crap, I can escape this bug by routing outgoing Invidious traffic to Tor. Just like this option on the good yt-local tool.

Piped now needs to facilitate the configuration of an outgoing upstream proxy, to the WARP network, to the Tor network etc.

If Youtube blocks centralized instances, we then need to decentralize the traffic.

[esme518]

I found this one can run in non-privileged-mode https://github.com/ArcticLampyrid/docker-warp-socks5

You can add this to the docker-compose file

    warp:
        image: qiqiworld/warp-socks5:latest
        restart: unless-stopped
        container_name: warp

Also add REQWEST_PROXY: socks5://warp:40000 to the config/config.properties file

Then you backend will use cloudflare WARP as the outgoing proxy.

[Astolfo2001]

I currently can't access any video via Piped because of the whole "Sign in to confirm that you're not a bot" BS.

[user-naa]

I am having this error in ALL instances. Sometimes some instances allow me to watch videos, but after several videos same error. I hope this will not be the end of Piped(((

[PestoPosta]

I to am having this issue.

[mycodedoesnotcompile2]

I found this one can run in non-privileged-mode https://github.com/ArcticLampyrid/docker-warp-socks5

You can add this to the docker-compose file

    warp:
        image: qiqiworld/warp-socks5:latest
        restart: unless-stopped
        container_name: warp

Also add REQWEST_PROXY: socks5://warp:40000 to the config/config.properties file

Then you backend will use cloudflare WARP as the outgoing proxy.

Thanks !

But "REQWEST_PROXY" or "REQUEST_PROXY" ? This is a standard env variable for setting outgoing proxy ?

[ngn13]

recently I've been experiencing this issue a lot, and I wrote a script to find available instances for a given video:

[500] https://pipedapi.kavin.rocks: Got error: "Sign in to confirm that you're not a bot"
[200] https://pipedapi.leptons.xyz: success
[500] https://pipedapi.r4fo.com: Got error: "Sign in to confirm that you're not a bot"
[200] https://pipedapi-libre.kavin.rocks: success
[500] https://pipedapi.in.projectsegfau.lt: Got error: "Sign in to confirm that you're not a bot"
[500] https://pipedapi.smnz.de: Got error: "Sign in to confirm that you're not a bot"
[500] https://pipedapi.astartes.nl: Got error: "Sign in to confirm that you're not a bot"
[500] https://pipedapi.drgns.space: Got error: "Sign in to confirm that you're not a bot"
[500] https://piapi.ggtyler.dev: Got error: "Sign in to confirm that you're not a bot"
[500] https://pipedapi.phoenixthrush.com: Got error: "Sign in to confirm that you're not a bot"
[200] https://schaunapi.ehwurscht.at: success
[500] https://pipedapi.darkness.services: Got error: "Sign in to confirm that you're not a bot"

so most the instances are currently unavailable due to this issue

also I want to mention that I don't think this is an IP blacklist issue, I tried using multiple different IPv6 and IPv4 addresses for my instance, after successfully watching one or two videos, the issue comes back again, I think youtube has a new way to verify actual clients/detect "bots", this is backed up by the fact that people having this issue while using NewPipe, are not having this issue when they visit the actual youtube website and try to watch the same video

[esme518]

I found this one can run in non-privileged-mode https://github.com/ArcticLampyrid/docker-warp-socks5 You can add this to the docker-compose file

    warp:
        image: qiqiworld/warp-socks5:latest
        restart: unless-stopped
        container_name: warp

Also add REQWEST_PROXY: socks5://warp:40000 to the config/config.properties file Then you backend will use cloudflare WARP as the outgoing proxy.

Thanks !

But _"REQWESTPROXY" or _"REQUESTPROXY" ? This is a standard env variable for setting outgoing proxy ?

Check the https://github.com/TeamPiped/Piped-Backend/blob/master/config.properties at line 12-16

# Outgoing proxy to be used by reqwest4j - eg: socks5://127.0.0.1:1080
#REQWEST_PROXY: socks5://127.0.0.1:1080
# Optional proxy username and password
#REQWEST_PROXY_USER: username
#REQWEST_PROXY_PASS: password

[mycodedoesnotcompile2]

recently I've been experiencing this issue a lot, and I wrote a script to find available instances for a given video:

[500] https://pipedapi.kavin.rocks: Got error: "Sign in to confirm that you're not a bot"
[200] https://pipedapi.leptons.xyz: success
[500] https://pipedapi.r4fo.com: Got error: "Sign in to confirm that you're not a bot"
[200] https://pipedapi-libre.kavin.rocks: success
[500] https://pipedapi.in.projectsegfau.lt: Got error: "Sign in to confirm that you're not a bot"
[500] https://pipedapi.smnz.de: Got error: "Sign in to confirm that you're not a bot"
[500] https://pipedapi.astartes.nl: Got error: "Sign in to confirm that you're not a bot"
[500] https://pipedapi.drgns.space: Got error: "Sign in to confirm that you're not a bot"
[500] https://piapi.ggtyler.dev: Got error: "Sign in to confirm that you're not a bot"
[500] https://pipedapi.phoenixthrush.com: Got error: "Sign in to confirm that you're not a bot"
[200] https://schaunapi.ehwurscht.at: success
[500] https://pipedapi.darkness.services: Got error: "Sign in to confirm that you're not a bot"

so most the instances are currently unavailable due to this issue

also I want to mention that I don't think this is an IP blacklist issue, I tried using multiple different IPv6 and IPv4 addresses for my instance, after successfully watching one or two videos, the issue comes back again, I think youtube has a new way to verify actual clients/detect "bots", this is backed up by the fact that people having this issue while using NewPipe, are not having this issue when they visit the actual youtube website and try to watch the same video

Thanks, very interesting !

Could you propose your script to: https://farside.link/ + https://github.com/benbusby/farside/

[ghost]

The Invisious team has found a solution: Bot error : https://github.com/iv-org/youtube-trusted-session-generator. https://github.com/iv-org/inv_sig_helper (More details): https://github.com/iv-org/documentation/pull/581 İnitial player error vs another error(old method): https://github.com/iv-org/smart-ipv6-rotator

[I-I-IT]

Surprised to not have seen this proposed : various apps (Grayjay being one) fetch the video from the iOS client to solve the issue. See : https://gitlab.futo.org/videostreaming/plugins/youtube/-/commit/be0e7891727986154f8482574ac02973fdbd4886

FreeTube fix : https://github.com/FreeTubeApp/FreeTube/issues/5502#issuecomment-2261544217

AFAIK, the issue here is more related to Google banning specific method of obtaining videos, not yet full IP ranges. The fix from invidious concerning this is here : https://github.com/iv-org/inv_sig_helper (here also talks about it https://gitlab.futo.org/videostreaming/plugins/youtube/-/commit/f57b7b330f566ccc4669fd9a62719601d8167228)

[ghost]

1. https://github.com/iv-org/invidious/commit/2d18ff1f80aa804a11108f346a7a3a20f4eb191d

2. https://github.com/iv-org/invidious/commit/2d7869b48ba1c1aca04fd66c52d37853321775cf They have already merged PR. Piped would fix it

[nikkon226]

recently I've been experiencing this issue a lot, and I wrote a script to find available instances for a given video:

[500] https://pipedapi.kavin.rocks: Got error: "Sign in to confirm that you're not a bot"
[200] https://pipedapi.leptons.xyz: success
[500] https://pipedapi.r4fo.com: Got error: "Sign in to confirm that you're not a bot"
[200] https://pipedapi-libre.kavin.rocks: success
[500] https://pipedapi.in.projectsegfau.lt: Got error: "Sign in to confirm that you're not a bot"
[500] https://pipedapi.smnz.de: Got error: "Sign in to confirm that you're not a bot"
[500] https://pipedapi.astartes.nl: Got error: "Sign in to confirm that you're not a bot"
[500] https://pipedapi.drgns.space: Got error: "Sign in to confirm that you're not a bot"
[500] https://piapi.ggtyler.dev: Got error: "Sign in to confirm that you're not a bot"
[500] https://pipedapi.phoenixthrush.com: Got error: "Sign in to confirm that you're not a bot"
[200] https://schaunapi.ehwurscht.at: success
[500] https://pipedapi.darkness.services: Got error: "Sign in to confirm that you're not a bot"

so most the instances are currently unavailable due to this issue also I want to mention that I don't think this is an IP blacklist issue, I tried using multiple different IPv6 and IPv4 addresses for my instance, after successfully watching one or two videos, the issue comes back again, I think youtube has a new way to verify actual clients/detect "bots", this is backed up by the fact that people having this issue while using NewPipe, are not having this issue when they visit the actual youtube website and try to watch the same video

Thanks, very interesting !

Could you propose your script to: https://farside.link/ + https://github.com/benbusby/farside/

This is the one that I wrote. Feel free to put in your favorite video handle:

#!/bin/bash

for i in $(curl -s 'https://piped-instances.kavin.rocks/' --compressed -H 'Accept-Encoding: gzip, deflate, br, zstd' -H 'Referer: https://piped.video/' -H 'Origin: https://piped.video'  | jq -r -c '.[] | .api_url'); do
    ret=$(curl "$i/streams/gUVLnK3kOK0" --silent | jq -r -c '.message')

    printf "%-50s %s\n" "[$i]:" "$ret"
done

[I-I-IT]

Just pointing out that this is indeed not an IP blacklisting issue. I tried using LibreTube with the Piped proxy disabled (meaning it directly connected to Google servers) on Wi-Fi, and I still got the "Sign-in,..." error.

My best guess would be that YouTube is detecting that requests come from the "Piped API".