iv-org / invidious

Invidious is an alternative front-end to YouTube
https://invidious.io
GNU Affero General Public License v3.0
16.23k stars 1.79k forks source link

[Discussion] How to circumvent YouTube blockage - #3822 #4045 related #3915

Closed unixfox closed 1 month ago

unixfox commented 1 year ago

You can discuss here the different ways to circumvent the blockage from YouTube.

Reminder, the official solution is in https://github.com/iv-org/invidious/issues/3915#issuecomment-1763203671

unixfox commented 1 year ago

Posting the comments from the previous issue.


Could it be useful to have a script that automatically changes the IPv6 address and hook it up to a cronjob?

Originally posted by @Ashirg-ch in https://github.com/iv-org/invidious/issues/3822#issuecomment-1573583623


Could it be useful to have a script that automatically changes the IPv6 address and hook it up to a cronjob?

There is a script just for that :D: https://github.com/ycd/ipv6-rotator

But I did not test if changing the IPv6 address in the same range fixes the issue or not.

Originally posted by @unixfox in https://github.com/iv-org/invidious/issues/3822#issuecomment-1573597311


This hit me hard. OVH only gives you a /128 block for the IPv6 space, so I could mitigate this for a while using that, but it would be pointless if I'm going to get IP blocked again on IPv6. :middle_finger: <---- that's for you Google

Originally posted by @trentwiles in https://github.com/iv-org/invidious/issues/3822#issuecomment-1585296609


If there's some sort of way to set up invidious with a SOCKS5 proxy, I could probably rotate IPs on DigitalOcean/Vultr.

Originally posted by @trentwiles in https://github.com/iv-org/invidious/issues/3822#issuecomment-1585297177


This hit me hard. OVH only gives you a /128 block for the IPv6 space, so I could mitigate this for a while using that, but it would be pointless if I'm going to get IP blocked again on IPv6. đź–• <---- that's for you Google

You could try getting a proper IPv6 range from tunnel broker such as Hurricane Electric.

Originally posted by @ddvarpdd in https://github.com/iv-org/invidious/issues/3822#issuecomment-1585487223


I've had success routing outbound invidious traffic over shared VPNs; blending traffic with 'legitimate' users and connecting from non-Hetzner/OVH/DO address space works well for now.

PostUp = ip route add default dev %i table 5280
PostUp = ip -6 route add default dev %i table 5280

PostUp = ip rule add from (wg v4 address) table 5280
PostUp = ip -6 rule add from (wg v6 address) table 5280
PostUp = ip -4 rule add uidrange 991-991 lookup 5280
PostUp = ip -6 rule add uidrange 991-991 lookup 5280

PostDown = ip rule del from (wg v4 address) lookup 5280
PostDown = ip -6 rule del from (wg v6 address) lookup 5280
PostDown = ip -4 rule del uidrange 991-991 lookup 5280
PostDown = ip -6 rule del uidrange 991-991 lookup 5280

Table = off

reCAPTCHA profiles addresses at the /64 so it probably won't be long until we see that on YT as well, if they aren't already.

Originally posted by @tempname1024 in https://github.com/iv-org/invidious/issues/3822#issuecomment-1586440214


I'm not smart when it comes to networking, where do you add these lines? Also, how do you authenticate with WG? You can't just connect to the IP without a private key right?

Originally posted by @trentwiles in https://github.com/iv-org/invidious/issues/3822#issuecomment-1586475722


I'm not smart when it comes to networking, where do you add these lines? Also, how do you authenticate with WG? You can't just connect to the IP without a private key right?

To keep wireguard from updating the system's default routing table we disable that functionality with Table = off, create our own table called 5280, add wireguard's addresses, and create a rule to lookup table 5280 for traffic which originates from the user running invidious, in my case the user ID is 991. These lines are added to the Interface section.

Originally posted by @tempname1024 in https://github.com/iv-org/invidious/issues/3822#issuecomment-1586593237


If there's some sort of way to set up invidious with a SOCKS5 proxy, I could probably rotate IPs on DigitalOcean/Vultr.

Double that. You could also use a “residential IP” proxy service, or even one of the mobile proxy ones (which use mobile operators networks and usually change IPs every couple minutes for you). This might not work well with proxying video traffic though, and I'm afraid it also might be not too cheap nowadays.

Originally posted by @notpushkin in https://github.com/iv-org/invidious/issues/3822#issuecomment-1588770313


If there's some sort of way to set up invidious with a SOCKS5 proxy, I could probably rotate IPs on DigitalOcean/Vultr.

Double that. You could also use a “residential IP” proxy service, or even one of the mobile proxy ones (which use mobile operators networks and usually change IPs every couple minutes for you). This might not work well with proxying video traffic though, and I'm afraid it also might be not too cheap nowadays.

Are you sure the video traffic has to be proxied at all?

Originally posted by @ccuser44 in https://github.com/iv-org/invidious/issues/3822#issuecomment-1589255884

notpushkin commented 1 year ago

@ccuser44 Only if you want to get resolutions greater than 720p – those are only available over DASH, and it only works with proxying: https://docs.invidious.io/geoblocking/#video-quality-and-dash

unixfox commented 1 year ago

@ccuser44 Only if you want to get resolutions greater than 720p – those are only available over DASH, and it only works with proxying: docs.invidious.io/geoblocking/#video-quality-and-dash

You need proxy for "encrypted" videos to load like music videos or so on, even when not using DASH.

zekefeu commented 1 year ago

If the GitHub repo has to go, we probably can make a Gitea instance as a Tor hidden service ? I'm willing to do some tests to see if and how it would work.

unixfox commented 1 year ago

If the GitHub repo has to go, we probably can make a Gitea instance as a Tor hidden service ? I'm willing to do some tests to see if and how it would work.

That's half related to the original topic which is circumventing the current youtube blockage not a potential future one. Don't worry we are deploying our services if one day the repository is taken down on github.

gi-yt commented 1 year ago

Welp google started ratelimiting IPv6 now (inv.bp.projectsegfau.lt). Surprisingly our main instance (invidious.projectsegfau.lt) is still running fine on v4 :P

unixfox commented 1 year ago

Welp google started ratelimiting IPv6 now (inv.bp.projectsegfau.lt). Surprisingly our main instance (invidious.projectsegfau.lt) is still running fine on v4 :P

Nice can you try to change the IPv6 address in another address in the same IPv6 range and see if it unblocks the instance?

tio-trom commented 1 year ago

Well....our https://ytb.trom.tf is also affected. Switching between ipv4 and 6 does not help anymore. My ipv6 ends in /64 so I guess I could change the IP address too? How will this affect my server overall? I have 20+ services hosted on that server.

Trit34 commented 1 year ago

https://invidious.fdn.fr/ blocked today, too. :triumph:

gi-yt commented 1 year ago

Changing IP made the instance unratelimited again :D

unixfox commented 1 year ago

Well....our ytb.trom.tf is also affected. Switching between ipv4 and 6 does not help anymore. My ipv6 ends in /64 so I guess I could change the IP address too? How will this affect my server overall? I have 20+ services hosted on that server.

yes change your IPv6 address like stated in https://github.com/iv-org/invidious/issues/3822#issuecomment-1565401621

if you have added your current IPv6 address in your domain record, then you will need to update it too.

tio-trom commented 1 year ago

Ok. But how long till Google will ban the new IPV6? Is this a solution / worth the effort?

unixfox commented 1 year ago

Ok. But how long till Google will ban the new IPV6? Is this a solution / worth the effort?

It seems like they ban almost all the invidious instances every 2 weeks. It's not a permanent solution, but it's a temporary one that works for the moment.

I want to reiterate on the temporary, we, the community, are trying to find a solution that works for the long term. Even though it's always a cat and mouse game with Google, there is no permanent solution.

notpushkin commented 1 year ago

Anybody gave (SOCKS / http) proxy support any more thought? Should we start an issue for that?

(I'll gladly look into implementing it, although I have absolutely zero experience with Crystal so it may take a while.)

llebout commented 1 year ago

Hello! In terms of bypassing IPv6 blocks, if you have a dedicated IPv4 you can register for a Free IPv6 Tunnel here: https://tunnelbroker.net/ - it's an Hurricane Electric service (big worldwide ISP)

Those can be as large as /48 however to avoid blocks maybe get a few smaller ones if possible (don't remember exactly how assignment works).

Additionally, you can rotate requests through a whole /48 block so that individual /64 blocks never reach the rate limit. However that would require a bit of advanced sysadmin and network code to achieve.

unixfox commented 1 year ago

Anybody gave (SOCKS / http) proxy support any more thought? Should we start an issue for that?

(I'll gladly look into implementing it, although I have absolutely zero experience with Crystal so it may take a while.)

There is an open issue for that: https://github.com/iv-org/invidious/issues/301


By the way useful and fun fact youtube is blocking per IP address even in ipv6. So if you have an ipv6 range of /64 available they will never be able to entirely block you because there are 18,446,744,073,709,551,616 IP addresses in a /64 ipv6 range.


Additionally, you can rotate requests through a whole /48 block so that individual /64 blocks never reach the rate limit. However that would require a bit of advanced sysadmin and network code to achieve.

We officially plan to provide dedicated tools for this.

tio-trom commented 1 year ago

By the way useful and fun fact youtube is blocking per IP address even in ipv6. So if you have an ipv6 range of /64 available they will never be able to entirely block you because there are 18,446,744,073,709,551,616 IP addresses in a /64 ipv6 range.

Yes but for example in my case, where I have a server with some 20+ services, changing the IPV6 all the time is never going to work since it will affect the other services. So it seems to me like hosting invidious is getting kinda impossible unless you use a VPS only for it.

unixfox commented 1 year ago

By the way useful and fun fact youtube is blocking per IP address even in ipv6. So if you have an ipv6 range of /64 available they will never be able to entirely block you because there are 18,446,744,073,709,551,616 IP addresses in a /64 ipv6 range.

Yes but for example in my case, where I have a server with some 20+ services, changing the IPV6 all the time is never going to work since it will affect the other services. So it seems to me like hosting invidious is getting kinda impossible unless you use a VPS only for it.

More details from the invidious public room:

we are going to soon share a solution with a tool that will automatically rotate the IPv6 address every day while retaining all your existing networking so that you can keep using a fixed IPv4 and IPv6 address.

tio-trom commented 1 year ago

we are going to soon share a solution with a tool that will automatically rotate the IPv6 address every day while retaining all your existing networking so that you can keep using a fixed IPv4 and IPv6 address.

Any update about this?

I tried to keep an eye on the matrix chat but was a bit too busy.

unixfox commented 1 year ago

we are going to soon share a solution with a tool that will automatically rotate the IPv6 address every day while retaining all your existing networking so that you can keep using a fixed IPv4 and IPv6 address.

Any update about this?

I tried to keep an eye on the matrix chat but was a bit too busy.

No I'll post the tool here so no need to remind us ;). My time is a bit limited lately.

ccuser44 commented 1 year ago

I myself use libredirect so i just switch the instance with Ctrl + Shift + L until I find an instance that works

unixfox commented 1 year ago

I have not forgotten about the tool, I just had some free time this weekend to think about it.

Basically, I've discovered that in ipv6, it's very easy to force an IP to use a certain IPv6 range. So I'm going to develop a tool that automatically use a new IPv6 address every day and force this new IP address only through Google official IPv6 subnets: https://md5calc.com/google/ip

This way no breakage of current setup, it's compatible with all the Linux systems and very easy to setup. It will just require to use IPv6 on Invidious and have IPv6 connectivity on the server. It's possible to use a tunnelbroker if the server does not have IPv6 connectivity: https://tunnelbroker.services/

In a nutshell, it's just:

ip addr add my:new:ipv6/64 dev eth0
ip -6 route add google:ipv6:range/37 via my:ipv6:gateway src my:new:ipv6

That's it.

EDIT: Have to investigate if this work for programs in docker containers.

unixfox commented 11 months ago

Hello to all public instances owners or people that are running a private Invidious instance that is blocked!

We have officially released a tool that allow to automatically rotate the IPv6 address of your Invidious server: https://github.com/iv-org/smart-ipv6-rotator and thus escape from the YouTube block!

You can follow the detailed tutorial here: https://docs.invidious.io/ipv6-rotator/

Please give your feedback, it is very important to us!


ping @tio-trom

joaquinito2070 commented 11 months ago

It works.

unixfox commented 11 months ago

We have fixed an issue where invidious would not send all the traffic in ipv6: https://github.com/iv-org/invidious/pull/4225 Please add this line in your config.yml to be sure that invidious send the traffic in ipv6:

force_resolve: ipv6

If you are using the rotator (https://docs.invidious.io/ipv6-rotator/), please make sure to have the line above and have updated Invidious!
We have reflected the documentation for this new change.

animegrafmays commented 9 months ago

youtube is blocking entire /64's doing this now btw

Trit34 commented 9 months ago

youtube is blocking entire /64's doing this now btw

Not problem on my favorite instance (for now).

unixfox commented 9 months ago

youtube is blocking entire /64's doing this now btw

Fortunately that is not true.

animegrafmays commented 9 months ago

youtube is blocking entire /64's doing this now btw

Fortunately that is not true.

fortunate for you maybe. if I disable naked IPv6 and use tunnelbroker then invidious doesn't lock up loading a video. that's a block.

unixfox commented 9 months ago

youtube is blocking entire /64's doing this now btw

Fortunately that is not true.

fortunate for you maybe. if I disable naked IPv6 and use tunnelbroker then invidious doesn't lock up loading a video. that's a block.

Please join our matrix for discussing more about it: https://matrix.to/#/#invidious:matrix.org

Github issues are not good for that.

Ghodawalaaman commented 4 months ago

is the issue still here?

unixfox commented 4 months ago

Youtube is still blocking the big instances so yes https://github.com/iv-org/invidious/issues/4045 hasn't been closed.

unixfox commented 1 month ago

Closing in favor of https://github.com/iv-org/invidious/issues/4734