feat: support external proxies

imputnet / cobalt

save what you love

https://cobalt.tools

GNU Affero General Public License v3.0

10.26k stars 722 forks source link

feat: support external proxies #588

Open teidesu opened 6 days ago

teidesu commented 6 days ago

added support for API_EXTERNAL_PROXY env variable that sets up a global undici dispatcher

definitely not the best way to do this (i really dislike relying on globals), but did it like this anyway because:

there isn't a single common wrapper over fetch() where i could provide a dispatcher
passing it manually everywhere is very tedious and error-prone (especially considering there aren't any types)
since api is supposed to run in a separate isolated process it probably won't affect what it isn't supposed to

one thing i worry about is the youtube handler which mentions reading dispatcher from streamInfo, but i couldn't find a place where it is being passed there 🧐

dumbmoron commented 5 days ago

one thing i worry about is the youtube handler which mentions reading dispatcher from streamInfo, but i couldn't find a place where it is being passed there 🧐

it's passed in from match.js if freebind is enabled (FREEBIND_CIDR has a value and syscall-napi compiled successfully).

it's only passed to some services that need it (youtube, twitter and instagram), because, as you mentioned, passing it manually everywhere is very tedious

dumbmoron commented 5 days ago

also it would be worth to mention that ProxyAgent only supports http(s)? proxies, so to fully support everything including using a socks proxy you would need to use something like fetch-socks

teidesu commented 5 days ago

only supports http(s)? proxies

yeah, and i mentioned it in the docs too. i personally don't think supporting socks is really worth it:

most of the (paid) proxy services provide both http and socks endpoints
most of the local software (e.g. xray-core and sing-box) have a "mixed" inbound that accepts both http and socks
if someone still wants to use socks proxy – they can use privoxy or whatever to convert it to http proxy

if one day undici ends up supporting socks (nodejs/undici#2224) we would get socks support for free :D

regarding the freebind dispatcher - i suppose we should error if freebind is enabled and proxy is passed, because there really isn't much point in using both of them at the same time