yggdrasil-network / yggdrasil-go

An experiment in scalable routing as an encrypted IPv6 overlay network
https://yggdrasil-network.github.io
Other
3.49k stars 239 forks source link

Define a specific standard for HTTP localhost proxies, or browser extensions, to enable advanced browser features? #745

Closed EternityForest closed 3 years ago

EternityForest commented 3 years ago

The full mesh encryption is great, but unfortunately, it seems that browsers do not care at all, and won't be treating Yggdrasil IP addresses as trusted anytime soon.

Perhaps a browser extension could be written, assuming no further nerfing happens, but a proxy also has a few other advantages, like working in any browser, like chrome mobile which lacks extensions.

I propose a standard for encoding both the port and IP into base64, and packing this into a subdomain of a proxy server, giving every mesh host a corresponding equivalent on a trusted localhost(On a random IP that's unlikely to conflict, like 127.5.9.12) but isolated with it's own subdomain.

This way, no self-signed certificate hackery is required to use the advanced features, many of which(like WebRTC and service workers) may be very important for a p2p future whee nodes spend a lot of time offline.

Especially if such a proxy was built directly into the main process itself, and widely accepted, it would make linking and sharing sites a lot easier.

It would also create an alternative to DNS that doesn't include any blockchains, which cause problems on limited storage. "Namefiles" could be shared that map from human readable localhost subdomains, to IPFS mesh names, similar to exchanging PGP keys.

This seems like something you'd want user-level control over, rather than using a system level daemon, but desktops are almost all single user anyway, and perhaps adding a new trusted site should involve the moment's pause that typing your sudo password gives(Or maybe it would only make people take sudo less seriously? Human factors is hard!).

The management app could eventually serve other purposes as well, like discovering and "pairing with" sites with things like QR codes that have the name mapping.

But in any case, the core functionality of HTTP proxying via subdomains would enable a lot of new use cases.

neilalexander commented 3 years ago

The main reason why Yggdrasil has never done anything regarding proxying like this is because, today, there's an expectation that most traffic carried over the Yggdrasil Network is indeed IPv6-encapsulated. If we add listeners for SOCKS proxying then we pretty much need to add the major workings of an IP stack into Yggdrasil to provide that encapsulation/decapsulation, otherwise the SOCKS node just won't be able to talk to anything else and that renders it largely useless.

This is far more complexity than we want to own and is quite far out-of-scope for the project.

cofob commented 3 years ago

Proxies do not need to process anything other than domains like <publickey>.yggip and ipv6 addresses. It can be used in conjunction with FoxyProxy for Firefox, for example.

neilalexander commented 3 years ago

The problem is that we'd need to implement an IPv6 stack and a TCP stack inside Yggdrasil in order for proxy nodes and IP nodes to talk to each other. Otherwise it's practically useless.

ccaapton commented 2 years ago

@neilalexander I also hope something like this to happen. The TCP stack could be done by leveraging tun2socks libraries, for instance . Would you have a look?

neilalexander commented 2 years ago

@neilalexander I also hope something like this to happen. The TCP stack could be done by leveraging tun2socks libraries, for instance . Would you have a look?

For what it's worth, I actually did some experimenting with the gvisor userspace networking stack and Yggdrasil — it's on the netstack branch and allows running Yggdrasil as a SOCKS proxy instead of using a TUN adapter. The problem is that it needs patches upstream and I don't have the time to commit to it (the gvisor code is a mess).

pufferffish commented 1 year ago

@neilalexander I also hope something like this to happen. The TCP stack could be done by leveraging tun2socks libraries, for instance . Would you have a look?

For what it's worth, I actually did some experimenting with the gvisor userspace networking stack and Yggdrasil — it's on the netstack branch and allows running Yggdrasil as a SOCKS proxy instead of using a TUN adapter. The problem is that it needs patches upstream and I don't have the time to commit to it (the gvisor code is a mess).

I'm looking for this exact feature and I was planning to make a pull request that uses netstack to let yggdrasil runs in userspace before I saw this comment. What exactly were the problems you ran into? I'm happy to contribute.

EternityForest commented 1 year ago

@pufferffish I never tried hacking on Yggdrasil, since I'm not a Go expert and generally don't bother working on things like that unless I think there's real interest, but I did implement the functionality I'm talking about in a simpler proof of concept:

https://github.com/EternityForest/hardlinep2p

This functionality is almost trivial these days, the big problem I ran into was mostly that my implementation was Python and didn't work well on Mobile, and I didn't see a whole lot of interest, so I never got around to making a Kotlin port, didn't seem worth it to maintain all by myself.

Plus, I was integrating it with a journaling and notes app, and I decided actually using a one-person DIY notes app wasn't what I wanted and moved to Keep.

The way my thing works is with TLS tunnels and OpenDHT. You hash the certificate (Not a real cert, It just generates one with dummy values), put that hash as an openDHT key(There's also multicast LAN discovery), and then on the client side, with no prior setup, you go to HASH.localhost:PORT.

The client parses the HTTP request, does a DHT lookup, verifes the cert, and you get a transparent tunnel.

Before that, the server sends a list of "Other IP Addresses you can find me at", so you can establish a connection on LAN, the server will tell the client it's WAN address, and then you can go on mobile and it will all still work even if something happens to OpenDHT, because it will just try the cached IP address for the node it's already seen.

I've been trying to get this functionality available for a while, because it would would make self-hosting things a lot easier and maybe even possible for mainstream users, but so far not much progress.

On Tue, Jun 13, 2023, 7:50 PM pufferfish @.***> wrote:

@neilalexander https://github.com/neilalexander I also hope something like this to happen. The TCP stack could be done by leveraging tun2socks libraries, for instance https://github.com/xjasonlyu/tun2socks . Would you have a look?

For what it's worth, I actually did some experimenting with the gvisor userspace networking stack and Yggdrasil — it's on the netstack branch and allows running Yggdrasil as a SOCKS proxy instead of using a TUN adapter. The problem is that it needs patches upstream and I don't have the time to commit to it (the gvisor code is a mess).

I'm looking for this exact feature and I was planning to make a pull request that uses netstack to let yggdrasil runs in userspace before I saw this comment. What exactly were the problems you ran into? I'm happy to contribute.

— Reply to this email directly, view it on GitHub https://github.com/yggdrasil-network/yggdrasil-go/issues/745#issuecomment-1590317241, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAFZCH2NWOXC7Y2PACZMKJLXLEKF7ANCNFSM4T4UWDGA . You are receiving this because you authored the thread.Message ID: @.***>