Open aschmahmann opened 5 years ago
Some notes I had:
/ipns/$key
refers to content and pathing, routing should be invisible when content addressing
/ipns/
links that resolve only over dht and don't work over pubsub ~/.ipfs/config
should have options to enable/disable resolvers, their priority, order, and tweak DoH server for DNS lookupsipfs name resovle
could have a --resolver dns|pubsub|dht
to enable ad-hoc diagnostics or resolving via specific methodQuestion around (delegated) lookup protocol:
Some thoughts on /ipns
addressing brought on by the above issues as well as:
ipfs/notes#348, ipfs/go-ipfs#6448, and ipfs/go-ipfs#3942
Naming resolution schemes have three components: 1) How do I identify my data? (public key, ASCII name, random 64bit integer) 1) How do I transport my data? (Server, DHT, PubSub, carrier pigeon) 2) What data format do I return? (IPNS record, DID Document, CID, IPLD Object, Raw binary, text)
IPNS (as specified in the spec) has the matching components: 1) IPNS Key (Multihash of a public key) 2) Transport (put/get) of IPNS record (DHT now, PubSub coming soon) 3) IPNS record Value, which is an arbitrary path
ipfs name resolve
you get a path, but a web-browser based dweb:/ipns/
results in a binary blob resolved from the path./ipns/DNSlink
confuses people and on the outset seems ridiculous because DNSlink does not support properties 1 or 2 of IPNS. However, it does actually satisfy the (somewhat inconsistent) property 3.
Putting dweb:/dns/
as it's own namespace makes a lot of sense since people tend to identify with properties 1 and 2 more than 3. However, we should make clear which of these properties is the one that merits getting a new namespace.
Some examples requirements for when to create new namespaces (and their implications):
dns
and ens
share a namespace, even worse all systems that use multihashes share a namespace (including ipfs
and ipns
)ipfs
, dns
, ens
get there own namespaces, but it's likely ipns-pubsub
and ipns-dht
get their own namespaces as wellipns
, dns
, ens
, ...
should all share the same namespace (ipfs
just barely gets it's own namespace because it never returns a path)Therefore, the Transport seems to be the only really useful piece of information in deciding whether or not to allocate a new namespace; new Transport => new Namespace.
If we really don't want ipns-dht
and ipns-pubsub
to be different transports (or at least to only be different transports) we can take @lidel 's suggestion above to the extreme and modify the IPNS protocol so that it is not router pluggable, but instead checks both the PubSub and DHT routers.
To be clear, any solution that says "check PubSub and if that fails check the DHT" is based on the fact that they share properties 1 + 2. However, DNS and ENS share those properties as well and so we could just have a /dns
namespace with ENS as a configurable routing mechanism and that would share properties with an IPNS with PubSub as a configurable router.
I agree that current IPNS+DNS situation at /ipns/
is unsustainable and extremely confusing.
At the same time, I am pretty sure "Transport" is not the best abstraction for drawing the line.
The purpose of content paths is to be transport-agnostic: we don't want to expose internals of IPNS resolver via /ipns
paths.
What if we frame the concept of namespaces as path Resolvers. Resolver takes a path and returns a path:
/ipfs
resolves to itself (ends resolving)/ipns
resolves libp2p-keys and returns a path/dns
resolves DNSLink over regular DNS and returns a path/ens
resolves "ENSLink" over EthDHT and returns a pathSome clarifying notes behind this way of thinking:
/ipns
paths don't care about IPNS backend.
Internals such as details of used "ipns transport" should be tweakable via ipfs config Ipns
Ipns.ResolveCacheSize
, adding things like Ipns.ResolverOrder
or Ipns.ResolverStrategy
feels like a safe way to go. /dns
don't care about which DNS server(s) were used, nor their caching strategies. DNS servers are set in OS and/or provided by ISP. /ens
, nor which EthDNS server was used. EthDNS servers can be tweaked via Ens.Nameservers
array.Other notes:
/ipns/{libp2p-key}
could point at /ipfs/
, /dns
or /ens
path. When accessing content using a path IPFS should resolve it recursively using a Resolver specific to the namespaces it discovers until it produces an immutable /ipfs/
path or hits the recursion limit.
This is how /ipns
works today, we would just extend it to multiple namespaces/ipns/{fqdn}
: it would be an alias for /dns/{fqdn}
, and in case of HTTP gateway produce the HTTP 301 Redirect to /dns
path@lidel Do you think you could clarify what makes DHT and PubSub routers of IPNS records similar, and yet DNS and ENS different?
I understand that IPNS over PubSub/DHT uses the same record type, but that's almost incidental. As a comparison think about the DID methods, even though they all have the same base record type the different transports/methods get different names did:ipid:bafyxyz, did:stack:me.stack, etc.
It's also worth noting that the transports inherently have security properties embedded in them which, while not relevant for immutable data/content-addressing like IPFS, are relevant with mutable data. For example, the IPNS PubSub router will allow third parties to persist IPNS records, but the DHT router will not.
I guess what I'm trying to do here is tease apart the difference between a Transport and a Resolver.
It's too high level to say "Resolvers map path => path (or sometimes path => data as I noted above), and every resolver gets its own namespace" because that means any two schemes that differ at all in the path => path process get their own namespace. That implies that because the DHT and PubSub routers of IPNS records do resolution differently from each other they should get their own namespaces (unless we define a protocol P that encompasses both routers). I tried Transport as a way of showing what our current plans seem to imply, but it doesn't have to be the only option. Any answer we come up with here could theoretically apply to the DID space as well, they seem to have settled on DID Method's as the implication of what needs a new namespace, but perhaps there are lessons we can learn.
clarify what makes DHT and PubSub routers of IPNS records similar, and yet DNS and ENS different?
In my mind IPNS remains a single namespace because while its internal "transports" can store and return different records for the same libp2p-key, the client can always pick and verify the latest record from across all available transports.
DNS<>ENS are not like this:
.eth
may produce conflicts when Ethiopia is added to DNSthink about the DID methods, even though they all have the same base record type the different transports/methods get different names did:ipid:bafyxyz, did:stack:me.stack, etc.
DIDs made me think about concepts of "Control" and "Equivalence". I feel those are useful in defining what content-addressed path namespaces are:
In my mind DID methods translate to sub-namespaces, because {foo}
in /did/ipid/{foo}
and /did/stack/{foo}
is not guaranteed to mean the same or to be controlled/managed by the same entity.
/ipns/{foo}
is different in that {foo}
means the same across all transports ({libp2p-key}
) and operates under guarantee of control by the same entity, no matter which IPNS transport/store you use.
@lidel @hugomrdias I was talking with @Stebalien about how we're currently using Pubsub + DHT as IPNS routers.
What do you think about publishing to all routing systems simultaneously, but allowing the Publish
function to return successfully as soon as even one routing system has sent out the message? This would make it so IPNS over PubSub users wouldn't have to wait around for a DHT publish to complete before ipfs name publish
returns.
@aschmahmann I'd be ok with such fast default, as long there is an opt-in way (eg. --wait-for dht,pubsub
parameter, or API to check status of pending publishing tasks) to wait for confirmation from specific system.
I imagine contexts where fast UI is built on top of backed that does not care about speed and wants to block until DHT publish succeeds.
Context
As noted in the IPNS spec there are currently two components of IPNS:
The spec currently does not specify the routing system by which a user that wishes to publish or resolve an IPNS record may do so, instead it says:
However, in actual IPNS implementations (e.g. in go-ipfs and js-ipfs) there is an implicit assumption that addresses such as
/ipns/$key
ordweb:/ipns/$key
will resolve in some standardized way. There's a dive into the world of dweb addressing by @lgierth and @lidel here.Thus far we've run into minimal problems here despite having both DHT and PubSub based resolution systems. This is because the PubSub system relies on the DHT for initial content resolution (i.e. only using PubSub for publishing/retrieving updates that occur while the daemon is running) and we always publish updates to both the DHT and PubSub. This effectively makes the DHT the authoritative store of IPNS information, with PubSub just being used for faster propagation.
Upcoming Issues
Some of the current endeavors to increase IPNS performance will involve creating record delivery mechanisms that are not reliant on using the DHT to propagate the IPNS records (e.g. a persistent PubSub channel, or IPNS over DNS). As a result, it's time for us to put together a proposal that adds some clarity to issues including:
/ipns/$key
?/ipns/pubsub-backed-by-dht/$key
or/ipns/$key-with-routing-info
)/ipns
like/ipld
refers to content and pathing, but not to record retrieval or propagation?Many of these issues were surfaced in a meeting on ongoing IPNS endeavors with notes at: ipfs/team-mgmt#948.
Thanks for your help and looking forward to hearing your thoughts: @Stebalien @pgte @satazor @vasco-santos @hugomrdias @jimpick @lidel @momack2