Accessing the Portal Network from a browser

[ComfyGummy]

tl;dr: Should the Portal Network be accessible from a web browser, with no additional software running on the local machine?

Longer version:

I am developing a Chrome extension to browse onchain websites using ERC-4804. Right now, it relies on centralized RPC providers like Infura/Alchemy to perform EVM calls to retrieve webpage contents. This has the obvious problems:

• Security: The client puts full trust in the RPC provider to return accurate data about the chain.
• Privacy: RPC providers get access to the user's browsing history; this is a similar problem as them getting the user's wallet addresses.
• Censorship: RPC providers can choose to selectively deny access to certain users or to certain onchain websites; this is a similar problem as them blocking transactions or denying access to certain block data.
• Centralization: Those RPC providers are centralized entities.

As I understand it, the Portal Network aims to address these challenges. I'd love to integrate it in my extension; it would create a true censorship-resistant, decentralized, open Web, all powered by Ethereum.

However, from my look into current implementations, all of them require software to be running on users' computers. For example, the Ultralight browser client requires a UDP proxy server to be running on a user's computer. From the perspective of a dapp developer, or a PWA developer, or in my case a browser extension developer, this makes the Portal Network impossible to integrate.

There could of course be a set of public UDP proxy nodes that Ultralight may connect to, but then it seems like that's only solving the "security" problem, and none of the other three (privacy/censorship/centralization).

I've noticed some discussion on this in issue #71 and issue #174 mentioning WebRTC as a possible transport, I'd also advise looking at the Tor Project's Snowflake effort which allows websites to embed a long-lived proxy within them that can act as entry points into the Tor network for others. I believe the technical problems they had to solve are quite similar as those facing the Portal Network.

But my question is higher-level: Should supporting purely-browser-based clients be a goal of the Portal Network? And if not, what type of client is the Portal Network meant to be integrated into?

Curious to hear your thoughts.

[acolytec3]

At one point, @scottypoi did implement a proof of concept using Waku/LIbp2p with a WebRTC transport layer to allow browser based Ultralight clients to discover and connect with each other but that never solved the issue of bridging the gap between browser clients and the underlying discv5 network (which is entirely UDP based). As you've noted, there have been a number of past discussions about how to allow browser clients to be first class users of the Portal Network but it really isn't feasible as long as we continue to leverage discv5 as the network/transport layer. Until the overall project moves to a different network stack (libp2p or something similar that offers full multiplexing across the lower level UDP/TCP/QUIC transport layers), it's likely not feasible to get truly independent browser clients.

[acolytec3]

As far as "what should" we support, that's a harder question. Browser based clients in their purest form aren't a boon for the network since they are ephemeral and don't really serve data to the network. If we could put them in extensions where they're longl-ived and can serve data back to the network and are generally online more, I think it makes a lot of sense but we haven't dug too much into that. I've always hoped we could at least put Ultralight in a mobile app where they could serve data but there are similar challenges with mobile apps being put to sleep and not being online much of the time.

So your question is a hard one to answer since we run into the same roadblocks as client developers for full ETH nodes

[ComfyGummy]

As pointed out in issue #71, browser extensions are also no longer allowed to be long-lived (at least in Chrome and Chromium-based browsers), largely from the same motivations.

I understand the point that clients that don't contribute resources back to a peer-to-peer network aren't really valuable from the perspective of that network. But at the same time, a peer-to-peer network isn't likely to grow if it doesn't meet users where they are.

Perhaps I'm misunderstanding the motivation for the Portal Network to begin with. If not browser clients, and if not mobile app clients, then who are the users that are intended to run Portal Network clients? Perhaps it is intended as a sort of "content delivery network" sitting in front of the main Ethereum peer-to-peer network? That would make sense and would definitely be valuable! But if so, then it feels like the Ethereum ecosystem is still missing an infrastructure piece for the "last-mile" delivery of EVM calls to end-user clients.

[qizhou]

I am wondering whether we could build the discv5 protocol over browser-friendly transport like websocket. This means that a browser (or a website with a portal network js client) can directly join the network, walk nodes in DHT, recursively find content in the network, and eventually serve the JSON-RPC requests from the front end.

The cons is that the transport overhead may be much larger than UDP, but considering the gain offered by the approach, I think it is worth more exploration.

I have a quick look at discv5 repo on sigp, and I think the implementation should not be hard if we emulate per UDP packet with a websocket connect/message/disconnect. This will definitively result in higher transport overhead. One further optimization may maintain a websocket connection pool so we can save the connect/disconnect cost for frequent communications of some specific nodes (e.g., the nodes in the routing table).

Has anyone done any exploration on this part? If not, I can take a further look and may come up with an MVP shortly.

[acolytec3]

Back when I started working on Ultralight, I built a minimal version that ran directly over websockets (so essentially one Ultralight node running in NodeJS running a websocket server and a second browser based client connecting to it via websockets and they exchanged discv5 messages as that way) but never explored it much further. I know at one point we briefly discussed having Trin/Fluffy also implement a websocket based solution so that browser clients could connect to multiple "peers" (i.e. Portal Network clients running a websocket server) directly but that line of exploration was never followed up on.

We also built a prototype of a mobile app that used the native device UDP connections to access the Portal Network (but it suffered the challenge of all mobile apps in that it would lose its network connection as soon as it went into the background or was put to sleep by the OS).

The current version of Ultralight runs in the browser by connecting the browser client to a "thin proxy" that is a simple websocket server that routes discv5 packets between the browser client and the discv5 network.

One other idea that @Scottypoi and I have kicked around but never really explored was using something like UnifiedPush to deliver discv5 via push notifications but have never put down any code on this idea.

The discv5 over WebRTC exploration that @Scottypoi did earlier this year was as close to true p2p as we ever got with browser clients but discovery was hard (we leveraged waku for that but that just basically bolts on a libp2p connection alongside webRTC). He had another idea for doing discovery by advertising the browser client webrtc connection offers using the discv5 REGTOPIC messages but here again, not explored in any depth.

I think we keep running into the same fundamental issue again and again that browser clients just require a proxy service of some kind to communicate with the network due to the underyling technology limitations, whether Ultralight's simple routing service or something more sophisticated like the libp2p network stack. That said, I know that I am always open and happy to discuss and explore new ideas!

[pipermerriam]

Some thoughts on supporting browsers.

Past experiments

The main past experiment that we did was bridging websockets <-> UDP. This "works" but I don't think it is actually a viable solution. The reason for this is that for the solution to work, the network needs enough capacity to bridge between the native UDP network and the browser websockets network. Since all of our packets are encrypted, the nodes that are bridging have no way of knowing if the traffic they are sending is legitimate discv5 traffic. This means that this bridging approach is susceptible to abuse.

There's enough fundamental problems burried in there that I'm not convinced that it's the right path towards a real solution.

The "right" solution

I believe that the correct solution for this will require something like https://veilid.com/

• The protocol that we use needs to formally support transports that are available for use within the browser.
  • websockets (I think this one has SSL issues)
  • webRTC (I think this one is the one that works best for browser P2P)
• The protocol needs to have built-in "seemless" relaying between nodes with incompatible transports

These features could in theory be brought into discv5, but more realistically, we might be required to change our base networking protocol in order to bring browsers into the game.... which puts this as a feature that we would need to both:

• decide it is important enough to do this significant change
• do the change soon enough that it's still viable to do it, but not so soon that it derails the timeline of the project.

At present, shipping a functional state network trumps this in priority because we need to demonstrate that portal based state network is viable and useful.

[qizhou]

Many thanks for the detailed information!

Back when I started working on Ultralight, I built a minimal version that ran directly over websockets (so essentially one Ultralight node running in NodeJS running a websocket server and a second browser based client connecting to it via websockets and they exchanged discv5 messages as that way) but never explored it much further. I know at one point we briefly discussed having Trin/Fluffy also implement a websocket based solution so that browser clients could connect to multiple "peers" (i.e. Portal Network clients running a websocket server) directly but that line of exploration was never followed up on.

That sounds to be the discv5 protocol over websocket I am proposing. I am glad that you have done the preliminary work and I would like to further move in this direction: implement the websocket transport for discv5 for Trin and run a much larger testbed (e.g., 100 nodes spread over 4-5 machines). This should give us more hints about the overhead of websocket over UDP. I think I could start with your code, and could you please point out where the code is?

[pipermerriam]

@qizhou I think this approach (discv5 over websockets) is known to work. The place where we have questions is how you bridge between the UDP <> websocket networks, otherwise you will have two networks that are effectively partitioned from each other.

[acolytec3]

Many thanks for the detailed information!

Back when I started working on Ultralight, I built a minimal version that ran directly over websockets (so essentially one Ultralight node running in NodeJS running a websocket server and a second browser based client connecting to it via websockets and they exchanged discv5 messages as that way) but never explored it much further. I know at one point we briefly discussed having Trin/Fluffy also implement a websocket based solution so that browser clients could connect to multiple "peers" (i.e. Portal Network clients running a websocket server) directly but that line of exploration was never followed up on.

That sounds to be the discv5 protocol over websocket I am proposing. I am glad that you have done the preliminary work and I would like to further move in this direction: implement the websocket transport for discv5 for Trin and run a much larger testbed (e.g., 100 nodes spread over 4-5 machines). This should give us more hints about the overhead of websocket over UDP. I think I could start with your code, and could you please point out where the code is?

I'm going to have to dig through git history to find it. It's been a long time since l touched that stuff.

[qizhou]

@qizhou I think this approach (discv5 over websockets) is known to work. The place where we have questions is how you bridge between the UDP <> websocket networks, otherwise you will have two networks that are effectively partitioned from each other.

I agree with that. My wild thought is that if we would put browsers as first-class customers of the portal network and the overhead of websocket/webrtc (and address SSL issue of course) is acceptable, we could opt for web-friendly transport instead of UDP and thus address the partition issue (and the proxy).

I understand that the decision may not be easy to make, but I believe that natively supporting web applications will be the unique feature for the portal network to stand out from existing P2P networks (e.g., IPFS or devp2p).

[pipermerriam]

@qizhou I think my previous statement was wrong.

• We do know that we can proxy websockets-or-webRTC -> UDP and make this work, but it isn't a viable solution.
• We do not know what it is like to actually run Discv5 over WebRTC-or-websockets

So if what you were proposing was to re-architect the client so that it natively does discv5 over a different transport, I think that there might be value in that.

[acolytec3]

• We do not know what it is like to actually run Discv5 over WebRTC-or-websockets

Ultralight has done both of these in limited capacities in the past. WebRTC was done here by @Scottypoi and my initial prototype for Ultralight included a complete, if primitive implementation of discv5 over websockets (though it seems to have been lost in our git history somewhere). I'm reasonably confident I could recreate it in a day or two leveraging the existing websocket transport we use for our browser implementation of discv5 (which currently only works as a web socket client so would simply need to add an alternative web socket server extension).

[pipermerriam]

I believe that the path for us adding browsers requires a transport supported by the browser since UDP is fundamentally not available in the browser.

Browser Supported Transport

Browsers support both websockets and webrtc as candidates for transports. However, in order for two browser clients to establish a websockets connection, one must act as server. In the browser context, if a user is on a domain that uses SSL, then the websockets server is also required to have an SSL server. A browser client is unable to attain a signed SSL certificate, and a self-signed certificate will result in the client getting security warnings that they would be required to click through. This eliminates websockets as a viable P2P transport

Leaving WebRTC...

It would be nice for someone to run an experiment with two browser based clients connecting and sending discv5 packets over a webrtc connection to see if there are any hidden problems....

Networking Protocol

Now that we know that we need WebRTC, we have to decide if we're staying on DiscV5 or moving to a different protocol that includes both UDP and WebRTC [Veilid](such as https://veilid.com/)

Converting everything to Veilid

This approach is dangerous to the project because it would likely result in more than 12 months of timeline slippage. I think this is a deal breaker...

Do it within DiscV5

To do WebRTC in discv5 we can:

• "Full WebRTC": convert all of our nodes to use webrtc
• "Multi Transport": upgrade discv5 to support multiple transports

Full WebRTC

I (piper) don't have enough expertise in WebRTC to evaluate some parts of this path

This approach would mean effectively forking the DiscV5 protocol and running our own version of it over WebRTC. This doesn't feel like a real solution since WebRTC is not a datagram protocol like UDP and thus we'd be introducing overhead into our protocol by wrapping discv5 packets into a WebRTC stream.... I suspect there's hidden problems in here.

Multi Transport

This approach involves successfully getting multi-transport support added to the discv5 protocol which ultimately would require buy-in from @fjl

Doing this would involve formalizing how discv5 packets would be transported over WebRTC, which is likely quite easy to do.

The harder parts are that we would have to decide how we are going to deal with communication between nodes with disparate transports.

The Disparate Transport Problem

Suppose our network is comprised of two transports A and B. This leaves three types of nodes.

• Nodes that only speak A
• Nodes that only speak B
• Nodes that can speak A and B (aka AB nodes)

In this model, there is no direct way for an A node and a B node to communicate. Let us consider what the network looks like in this situation.

From the perspective of an A node, none of the B nodes would be contactable. If an A node sends a FINDNODES request to an AB node, they will likely get back some B nodes in the response, all of which would be unreachable. If the network is majority B nodes, then:

• A nodes would find themselves effectively on a tiny network comprised only of A and AB nodes, and much of the information they get from AB nodes would be useless because it would contain mostly B nodes which are unreachable from A nodes.
• AB nodes would have the best view of the network, though, they would consistently be providing both A and B nodes with ENR records for nodes that they cannot contact.
• B nodes would largely operate just fine, though they would have some small negative performance issues from AB nodes providing them with A nodes in their response data.

It is not clear how to quantify the extent of these problems, but they produce an environment in which degenerate network conditions are likely and thus, is probably not a viable solution to have disparate transports with no way for A and B nodes to communicate.

Message Relaying (aka Routing)

The natural next step for this is to have the AB nodes relay messages between A and B nodes.

This however is not a simple thing to implement securely as well as also still having a degenerate mode when there are not enough nodes willing to act as relays.

Further reading: https://veilid.com/docs/private-routing/

[pipermerriam]

goal of my above comment is to try and capture the problem statement... which the TLDR is that browser support isn't easy and will likely require significant work no matter the approach we take.

[weipin]

Any particular reasons why running additional software isn't considered a viable option? For Desktops, while inconvenient, it can be reasonable or even should be encouraged.

Benefits:

1. Stand-alone programs tend to be run longer, better contributing to the Portal Network.
2. A stand-alone program behaving as an RPC provider shares the same data with all external apps. On the other hand, when an implementation is 'embedded,' I suppose each 'instance' has to go through the same steps, such as joining the network, finding nodes, fetching data, etc.

Other random thoughts:

1. Categorize user cases into two classes: long-running and short-running. Long-running cases are instances run on "servers" or Desktops. Short-running cases are ones run on mobile devices. For the latter, I am afraid adjustments have to be made to reflect the fact that apps on phones cannot/shouldn't run long-time background tasks.

   The point is that nodes aren't equal - it makes little sense to include a (mobile phone) node in routing tables, and then find the node disconnected in a few minutes.

2. For Desktops, encourage users to install and run stand-alone programs. Make the programs easy to launch and run as a background process. Create web pages and installing packages that third parties can point to.

[qizhou]

Any particular reasons why running additional software isn't considered a viable option? For Desktops, while inconvenient, it can be reasonable or even should be encouraged.

I admit that the lower cost of running a portal network node compared to an Ethereum node will significantly lower the barrier, but I am afraid that running additional software will be inconvenient for most existing web users.

[qizhou]

goal of my above comment is to try and capture the problem statement... which the TLDR is that browser support isn't easy and will likely require significant work no matter the approach we take.

I did a quick experiment on top of WebRTC to build a data channel between a webpage and a python3 code here https://github.com/qizhou/research/tree/main/websocket/webrtc, and I believe the WebRTC on a browser P2P may still require centralized servers for brokering connections.

The good news is that by reusing existing libraries (e.g., https://github.com/feross/simple-peer and aiortc), writing a transport using WebRTC is not hard. However, the core issue is that unlike UDP, WebRTC's offer/answer connection negotiation protocol needs a centralized server to coordinate - as illustrated by the demo, to establish the data channel, both peers need to send an offer packet and an answer packet via a control or signal channel. Although the signal channel can be anything, for web apps, the channel must be accessible in a web browser, and thus the easiest way is to use centralized server via a secure HTTP or websocket transport.

I also take a look at some existing famous protocols that uses WebRTC (or claim to use WebRTC) to build P2P network:

• [Veilid](such as https://veilid.com/), where the website says "Protocol support is extensible and may add WebRTC and other specialized protocols in the future."
• WebTorrents, which is based on simple-peer but its DHT over WebRTC is not available in the browser (here is an old opening issue for that https://github.com/webtorrent/webtorrent/issues/288).
• PeerJS https://peerjs.com/, which simplifies WebRTC in JS but still needs a PeerServer to broker connections.

In summary, I believe that WebRTC is a promising solution for accessing P2P network without extension / plugin in a browser, but we have to address or accept some fundamental issues, especially the centralized broker server.

[acolytec3]

In our WebRTC explorations, we used libp2p gossip mechanics (specifically over waku but you could use any gossip strategy) to broadcast the WebRTC offer/accept messages and did discovery that way. This required probably too much additional code but the signal channel was still p2p. One way we could potentially reuse discv5 here is to broadcast peers WebRTC offer info using the discv5 REGTOPIC and associated responses info to share NODES type details with other peers. We'd need to do this in concert with some additional key in the ENR indicating a node uses WebRTC or something similar since WebRTC offers won't fit in the ENR

[pipermerriam]

@qizhou forgive this slightly short answer to a somewhat involved question.

You are correct that people running a desktop version of portal that the browser connects to is functional, and that long live desktop versions of the client running would be a more robust architecture for overall network health.

However, my experience is that people will not install and run desktop software other than in rare situations, and I'm not convinced that we would be successful in getting people to install and run our clients...

[qizhou]

However, my experience is that people will not install and run desktop software other than in rare situations, and I'm not convinced that we would be successful in getting people to install and run our clients...

Hi @pipermerriam. I fully agree that running desktop software for people to access Ethereum JSON-RPC is not a viable solution. One tricky case is whether we may allow users to install a browser extension to access the portal network. Especially, considering most of the users already use web3 wallets, if we could persuade MetaMask or other widely-used web3 wallet extensions to support the portal network (or develop a MetaMask plugin as MetaMask Snaps here https://metamask.io/snaps/), that may solve the transport issue easier (e.g., bypassing SSL).

[pcfreak30]

Hello, I came across this issue while doing research on ETH and thought I would give my 2C.

I have been spending nearly 2 years now solving the "web3 access" issue broadly, not just for ETH. During that time ive more/less experimented with most things discussed here.

My own project Im building solves some of this by using a tor-like websocket proxy -> DHT using https://github.com/holepunchto/hyperswarm-dht-relay which uses hyper/dat at the networking stack.

In the future I expect I can get WebRTC involved as well.

Now this relates to the current debate, because the reality is, you either need a proxy, even if it is a decentralized/P2P one, or you need to be able to talk to bootstrap nodes direct over websocket/webtransport.

As for dedicated software, my perspective is this is eventually going to be required. I actually released a prototype Firefox extension to avoid extra s/w, because it allowed things Chrome does not, and it was DOA (no one cares about Firefox). Honestly for browsers, the true end game is going to end up as a browser fork that can support everything web3 networks need. Anything trying to fit into the current web2 sandbox is more/less DINO until we can evolve web3. Hell even metamask is basically centralized right now, in exchange for wide adoption.

So, if you can enable direct web-friendly transports... you will save a lot of engineering pain, otherwise your kicking the can to need a separate relay P2P network, which means extra dev ops burden and cost, to blind proxy the data for browsers. From my POV, web3 is for web access and so decentralizing how browsers CAN access the ETH net censorship-free, generally should be a priority.

Kudos.

[pcfreak30]

Also @ComfyGummy you may be interested in my project since we are doing somewhat similar things. See https://lumeweb.com.

If you want to talk, you can join the discord linked at the bottom. Always open to collaborating.

Kudos!

[fjl]

However, my experience is that people will not install and run desktop software other than in rare situations, and I'm not convinced that we would be successful in getting people to install and run our clients...

@pipermerriam

One way I would personally envision access to Ethereum, is as a VPN-like app. Users already know that, in order to access a VPN one has to install the software for it and connect it. The concept of a system tray utility application is old enough that even the least computer-savvy people understand it now.

The other thing is that access to the chain will probably always be mitigated by a wallet app. So it would make sense to bundle a Portal client with a desktop wallet, and design the experience such that it's an always-on application. People need keys and key operations, they want to see their transactions, etc. so they will see a purpose in the app.

[fjl]

Regarding discovery in the browser, I think it would be interesting but likely also a bit different. It should kind of work since ENRs are extensible and one can publish any kind of ENR in the discv5. Nodes which are able to connect to both networks can bootstrap the discv5 table from either one because they just need to find a couple of ENRs with a working UDP endpoint. It doesn't really matter where they find them.

Building a DHT with forwarding where browsers are involved will lead to very high latency. I think it's a dead end. If you want to give direct access to browsers, you need to ensure they will find servers with a decent uplink, and make a direct connection to them without an intermediary.

[pipermerriam]

I'm leaning more and more towards base layer needing to have transport that is directly accessible to browsers, which seems to basically mean we have to build a network out that uses WebRTC as it's base P2P transport layer.

• websockets result in SSL issues
• routing/relaying/proxying introduces significant latency overhead

[pcfreak30]

I'm leaning more and more towards base layer needing to have transport that is directly accessible to browsers, which seems to basically mean we have to build a network out that uses WebRTC as it's base P2P transport layer

If you do that, you will probably end up needing a relay network in some form that handles signaling. I will be going out of my way to not rely on google STUN/TURN, and I have a route to getting this done in my project in the future, using a DHT WS proxy to exchange session data.

WebRTC largest downfall is NAT and things failing requiring you to proxy anyways.

[pcfreak30]

The other thing is that access to the chain will probably always be mitigated by a wallet app. So it would make sense to bundle a Portal client with a desktop wallet, and design the experience such that it's an always-on application

Short term I can agree with this, long term I think network access will need to be abstracted out and be lower level.

[pipermerriam]

I'm curious if our planned "rendezvous" protocol in DiscV5 would be sufficient for WebRTC NAT traversal issues.

With a sufficient number of non-NAT'd nodes, it seems that Rendezvous would allow the NAT'd nodes to hole punch. But I suspect it won't be "smooth" and that there'd be a decent number of connectivity issues that could arise in certain network conditions.

[pcfreak30]

I'm curious if our planned "rendezvous" protocol in DiscV5 would be sufficient for WebRTC NAT traversal issues.

With a sufficient number of non-NAT'd nodes, it seems that Rendezvous would allow the NAT'd nodes to hole punch. But I suspect it won't be "smooth" and that there'd be a decent number of connectivity issues that could arise in certain network conditions.

You might look at both libp2p and hyper/dat to see how they are dealing with hole punching, it is out of scope for my focus right now as im building layers ontop of these systems, not the systems themselves. Im focusing higher level.

But I would definitely use some P2P network to do signaling for WebRTC and avoid web2 infra.

[pipermerriam]

@pcfreak30 this? https://docs.holepunch.to/quick-start

[pcfreak30]

@pcfreak30 this? https://docs.holepunch.to/quick-start

Yes hyper/dat https://dat-ecosystem.org/ is basically another libp2p. They are currently funded by Tether.