Node reuse and resource sharing on the web

[lidel]

This is a placeholder for tracking open problems and related issues / notes

Embedding js-ipfs on a page binds it to specific tab, origin, and storage.

Open problems

Per Origin

js-ipfs embedded on a web page runs in the context of page's Origin, which means:

• datastore uses local storage allocated to the origin, this means permissions and storage limits are shared with other JS running on a page
• opening a new tab for the same page spawns a new, full instance of js-ipfs operating on the same datastore
  • :green_heart: ipfs-message-port modules now enable node sharing withing the same Origin: https://github.com/ipfs/js-ipfs/issues/3022

Cross Origin

Each websites running js-ipfs does it on its own:

• new datastore is created per Origin
  • nodes are unable to share storage,
  • data for a single CID can be stored multiple times in the same browser profile

Same machine

js-ipfs is unable to discover ipfs-desktop running on the same machine. It could leverage its existence for:

• delegated routing
• content preload
• circuit switching (relay)
• shared cache, data deduplication
• persistence (ensuring CIDs are cached outside non-persistent browser storage)

References

• Web API: "best-effort" vs "persistent" storage
• libp2p/specs/relay
• delegated routing: https://github.com/ipfs/js-ipfs/issues/2155 / https://github.com/libp2p/js-libp2p/issues/120

Prior Art

• multiple tabs leader election
  • https://github.com/jimpick/multiple-tab-logux/
• Exploration: P2P Network access through the service worker
  • PoC: https://github.com/Gozala/lunet/
  • https://gozala.io/work/lunet

[dokterbob]

Hi @lidel, others, this issue has been a big blocker in terms of usability for us to run js nodes client-side on ipfs-search.com

Is any work on this being done? Is this on the roadmap at all?

https://github.com/ipfs-search/dweb-search-frontend/issues/150

[lidel]

There are some recent developments which make it easier to work with content-addressed data, often removing the need for running full js-ipfs, or hardcoding some WebSockets providers, preload nodes for performing bitswap:

• delegated routing
  • We now have reframe specification, it allows for creating implementation-agnostic routing systems. It is already supported by Kubo, you can add additional reframe routers to the config, I believe js-ipfs will switch to it at some point in near future
• trustless retrieval
  • you no longer need to run full js-ipfs to fetch CID data and verify hashes.
  • trustless gateways allow you to fetch raw block or a bag of blocks and deserialize them into files see tools like ipfs-car for code examples.
• WebTransport
  • it solves the problem of WebSockets requiring TLS cert, which is limiting the connectivity options on the web (js-ipfs running on a web page can't dial TCP and QUIC peers)
  • wip spec in https://github.com/libp2p/specs/pull/404/files – see /certhash section to understand why this is a big deal for us

[RangerMauve]

Is using SharedWorkers under the hood an option? It think one limiation there is that stuff like WebRTC wouldn't be able to run within the worker.

[Gozala]

Is using SharedWorkers under the hood an option? It think one limiation there is that stuff like WebRTC wouldn't be able to run within the worker.

That is accurate. Thinking around there had been that we could implement custom transport that would leverage WebRTC in the host frames (select arbitrary host and ask it to proxy messages, if host times out select next host etc...).

It is also worth calling out that SharedWorker was deliberately not implemented by Safari, however I've done some experiments simulating SharedWorker API with ServiceWorker instead.