search

iv-org / invidious

Invidious is an alternative front-end to YouTube
https://invidious.io
GNU Affero General Public License v3.0
16.49k stars 1.86k forks source link

[Bug] Server ip address leaked when behind a proxy. #2142

Open moom0o opened 3 years ago

moom0o commented 3 years ago

Describe the bug Bad actors can easily get the backend server ip by logging the web requests. This in turn can allow bad actors to easily ddos the backend server.

Steps to Reproduce

  1. Click on a random video
  2. Press F12
  3. Go to network
  4. Click play
  5. Find a web request to googlevideo.com
  6. The ip is located in the request url, making cloudflare/other ddos protection services useless.

Screenshots Image Another request Additional context

Is there any important reason why the server ip address needs to be sent in a request, or can this easily be removed? Thanks.

unixfox commented 3 years ago

If I'm not mistaken, this is needed for the videoplayback endpoint, by removing it the YouTube server will drop the request.

I just tried a random URL from invidious.snopyta.org:

https://invidious.snopyta.org/videoplayback?expire=1623354459&ei=-xfCYL2YNsKBpASgy5_AAg&ip=95.216.24.230&id=o-AGqQwXRYsJP54m4WpqotuTFmTcooej9JwbV9fxWJgS4k&itag=135&aitags=133,134,135,136,160,242,243,244,247,278&source=youtube&requiressl=yes&mh=0S&mm=31,29&mn=sn-i5heen7s,sn-5goeen76&ms=au,rdu&mv=m&mvi=4&pl=24&initcwndbps=581250&vprv=1&mime=video/mp4&ns=VWWrzUuCWRgT1PT_NNzf59wF&gir=yes&clen=34352697&dur=450.299&lmt=1623327028914041&mt=1623332214&fvip=6&keepalive=yes&fexp=24001373,24007246&c=WEB&txp=6316222&n=Mzoj-asFczM1lUur&sparams=expire,ei,ip,id,aitags,source,requiressl,vprv,mime,ns,gir,clen,dur,lmt&sig=AOq0QJ8wRQIhALLf8KKLjK2q9Lfgp3pyQEU7gIv91FAUTlekJYu8wa3rAiAhKIO4Hfgy5VPT_pQscrvWuGoC-wzxX6AcZHbMESQlZA==&lsparams=mh,mm,mn,ms,mv,mvi,pl,initcwndbps&lsig=AG3C_xAwRAIgSuYrCXMbbhjZkbXsX4luuD3VWJmV8c6h7RIxICnT6sgCIEGrLtVEc3wZnrOa6x-WqsqND8cF5K_j3nUJdvFPaTMz&host=r4---sn-i5heen7s.googlevideo.com

Works

But not when ip= is omitted: https://invidious.snopyta.org/videoplayback?expire=1623354459&ei=-xfCYL2YNsKBpASgy5_AAg&ip=95.216.24.230&id=o-AGqQwXRYsJP54m4WpqotuTFmTcooej9JwbV9fxWJgS4k&itag=135&aitags=133,134,135,136,160,242,243,244,247,278&source=youtube&requiressl=yes&mh=0S&mm=31,29&mn=sn-i5heen7s,sn-5goeen76&ms=au,rdu&mv=m&mvi=4&pl=24&initcwndbps=581250&vprv=1&mime=video/mp4&ns=VWWrzUuCWRgT1PT_NNzf59wF&gir=yes&clen=34352697&dur=450.299&lmt=1623327028914041&mt=1623332214&fvip=6&keepalive=yes&fexp=24001373,24007246&c=WEB&txp=6316222&n=Mzoj-asFczM1lUur&sparams=expire,ei,ip,id,aitags,source,requiressl,vprv,mime,ns,gir,clen,dur,lmt&sig=AOq0QJ8wRQIhALLf8KKLjK2q9Lfgp3pyQEU7gIv91FAUTlekJYu8wa3rAiAhKIO4Hfgy5VPT_pQscrvWuGoC-wzxX6AcZHbMESQlZA==&lsparams=mh,mm,mn,ms,mv,mvi,pl,initcwndbps&lsig=AG3C_xAwRAIgSuYrCXMbbhjZkbXsX4luuD3VWJmV8c6h7RIxICnT6sgCIEGrLtVEc3wZnrOa6x-WqsqND8cF5K_j3nUJdvFPaTMz&host=r4---sn-i5heen7s.googlevideo.com

The result is 403 forbidden HTTP error.

Usually this is not really an issue because the backend IP address is the same as the one that publicly serve the traffic for the users.

But I get your point, how would we proceed so that the videoplayback endpoint still works while hiding the IP address?

SamantazFox commented 3 years ago

Except by forcing video proxying through your instance, no, there is no way to do so. The IP is part of the parameters and removing it or spoofing it breaks the signature. It would require a lot of reverse engineering to understand how all those things work, and if the signature is some high-end crypto stuff, this will he impossible to crack.

unixfox commented 3 years ago

Except by forcing video proxying through your instance, no, there is no way to do so. The IP is part of the parameters and removing it or spoofing it breaks the signature. It would require a lot of reverse engineering to understand how all those things work, and if the signature is some high-end crypto stuff, this will he impossible to crack.

Well we could just save the IP address in memory and insert it for every request instead of passing it through the client browser, this way we break nothing.

moom0o commented 3 years ago

Except by forcing video proxying through your instance, no, there is no way to do so. The IP is part of the parameters and removing it or spoofing it breaks the signature. It would require a lot of reverse engineering to understand how all those things work, and if the signature is some high-end crypto stuff, this will he impossible to crack.

Isn't video already being proxied? How would I force this?

Perflyst commented 3 years ago

There is a toggle in the Invidious settings Proxy videos:. You can enable it by default on your instance with

default_user_preferences:
  local: true
SamantazFox commented 3 years ago

Well we could just save the IP address in memory and insert it for every request instead of passing it through the client browser, this way we break nothing.

@unixfox What do you mean? If we don't provide the direct URL to the client (so they can connect directly to Youtube servers), we must proxy this through the instance.

moom0o commented 3 years ago

There is a toggle in the Invidious settings Proxy videos:. You can enable it by default on your instance with

default_user_preferences:
  local: true

The IP is still here. image

unixfox commented 3 years ago

Well we could just save the IP address in memory and insert it for every request instead of passing it through the client browser, this way we break nothing.

@unixfox What do you mean? If we don't provide the direct URL to the client (so they can connect directly to Youtube servers), we must proxy this through the instance.

I mean, we omit the ip parameter from the query URL when generating the HTML page on the frontend, but we insert it back when we pass the request to the YouTube servers.

SamantazFox commented 3 years ago

I mean, we omit the ip parameter from the query URL when generating the HTML page on the frontend, but we insert it back when we pass the request to the YouTube servers.

In the case where local is not enabled, this won't change anything. Anybody opening the devtools/inscpector will be able to see it.

unixfox commented 3 years ago

I mean, we omit the ip parameter from the query URL when generating the HTML page on the frontend, but we insert it back when we pass the request to the YouTube servers.

In the case where local is not enabled, this won't change anything. Anybody opening the devtools/inscpector will be able to see it.

Invidious is in control of what URL are being generated for the client, we "just" have to hide this ip parameter from any URL query that we give to the client.

I know this won't be possible for the googlevideo.com URL, but in this case we can offer the ability to force the usage of proxy videos through the invidious server. In this case, we can always manipulate the URL and hide the IP address.

SamantazFox commented 3 years ago

Invidious is in control of what URL are being generated for the client, we "just" have to hide this ip parameter from any URL query that we give to the client.

I know this won't be possible for the googlevideo.com URL, but in this case we can offer the ability to force the usage of proxy videos through the invidious server. In this case, we can always manipulate the URL and hide the IP address.

ah, yeah. It will need another server config option, something like force_video_proxying or hide_server_ip...

unixfox commented 1 year ago

One idea that popped into my mind, encrypt the ip= parameter using the hmac_key when passing it to the client, then decrypt it on the server side when the client sends back the query.

With the upcoming requirement of using hmac_key this could easily be feasible: https://github.com/iv-org/invidious/issues/3854

The downside is that this will break alternative proxies like http3-ytproxy: https://github.com/TeamPiped/http3-ytproxy