ytdl-org / youtube-dl

Command-line program to download videos from YouTube.com and other video sites
http://ytdl-org.github.io/youtube-dl/
The Unlicense
129.72k stars 9.78k forks source link

Randomly select an available IP address for --source-address (was: ipv6 selection) #9601

Open Bryannn opened 8 years ago

Bryannn commented 8 years ago

I see there is an option --force-ipv6 to use ipv6, does it choose a random ipv6 ip from my range ?

Thanks!

yan12125 commented 8 years ago

It chooses "the best match" IPv6 address on your device, which depends on your routing table.

Bryannn commented 8 years ago

I asked this because my ipv4 got blocked by YouTube for sending too many requests. So now I don't want it to use just 1 IP for downloading. If it chose a random IP out of the 256 IP's, would have been great. what do you exactly mean by best match. Could the best match keep changing each time it downloads? Thanks!

Bryannn commented 8 years ago

is there a way to know which ip is using ? cant see the used ip in the verbose mode

yan12125 commented 8 years ago

what do you exactly mean by best match.

See https://en.wikipedia.org/wiki/Routing_table.

Could the best match keep changing each time it downloads?

In general no.

yan12125 commented 8 years ago

is there a way to know which ip is using ?

Add --call-home

And by the way, there are several "too many requests" reports and others' solution may help: #5138 and this FAQ entry

Bryannn commented 8 years ago

I really appreciate all your help. I've already read that post. I just have one last small question. So what if the best match IP gets blocked by YouTube for sending too many requests. Will it move to another IP from the range or just stop there? Thanks :)

yan12125 commented 8 years ago

It will just get blocked. youtube-dl is not intelligent enough to know the next unblocked IP :)

Bryannn commented 8 years ago

Can someone here help me change the code of youtube-dl so that it chooses a random IP each time from my ipv6 range instead of just using the best one. I could pay for it.

yan12125 commented 8 years ago

FYI: you can use --source-address to force the IP to use. You may need to write a script to choose from available IPs and assign it to youtube-dl.

Bryannn commented 8 years ago

Yes that's what i'm talking about. If some here could help me do that.

yan12125 commented 8 years ago

Here's an issue: usually YouTube blocks a subnet (an IP range) rather than a single IP, so random selection may not be useful.

siddht4 commented 8 years ago

what i suggest you guys is to rather connect it to a vpn or proxy to download the same, whatever you do your actual ip broadcasted will be that of the ISP which will not change. You might even get a warning to use your bandwidth fairly. Just report youtube that someone else on your network misused it and you are not involved . selecting random ipv6 will also be a security issue and misuse of the entire network.

My suggestion wait as youtube will white list you soon (unless it thinks it to be a ddos attack) , and use any proxy or vpn for a while

Bryannn commented 8 years ago

What I was thinking was. Let's say I'm sending 10K requests a day. If it chose the best IP then that IP is sending 10K requests and will get blocked super quick. But if it chose a random IP, then 10K requests get divided by 256 that is 39 request per IP. Hence it might not get blocked. So that is why I think this random selection will be a good feature.

yan12125 commented 8 years ago

I guess YouTube (and possible many other websites) counts requests by subnets rather than individual IPs, as changing IP is so easy. The hypothesis is easy to test: once an IP is blocked, change to another IP with --source-address.

Bryannn commented 8 years ago

--source-address will work for ipv6?

yan12125 commented 8 years ago

Yes. In fact --force-ipv6 is a shortcut to --source-address ::

Bryannn commented 8 years ago

Alright then. Thanks a lot for your help. Highly Appreciated!

Bryannn commented 8 years ago

Yan can we talk privately?

yan12125 commented 8 years ago

You can drop me an email.

Bryannn commented 8 years ago

Hi Yan, I just sent you an email with this subject: Randomly select an available IP address for --source-address

Please check.

Thanks!