IPNS Pubsub Reprovider Duration (GC for unused topics)

lidel commented 2 years ago

TLDR

Part of https://github.com/ipfs/go-ipfs/issues/6621, high impact: unbocks enabling fast IPNS, pubsub, and mitigating IPNS link rot by reproviding records

Rationale

We want go-ipfs users in desktop use case (IPFS Desktop, Brave) and HTTP Gateways to have Pubsub and IPNS over pubsub enabled by default (#6621).

One of the blockers for enabling it by default for all our users is IPNS over pubsub listening to IPNS topic forever. When running on a server hosting HTTP Gateway (or a very active desktop client) resolving /ipns/{libp2p-key} will add topic listener that never expires.

Proposed fix

Introduce usage-based GC for unused pubsub topics created by IPNS over pubsub.

We don't want the expiration to be too aggresive: there is a value in peers running "IPNS record reprovider" over the libp2p's fetch protocol. I think ~48h would match current expiration window of IPNS record in DHT + solve the problem of "IPNS rot" caused by someone's laptop going offline.

To implement the GC:

go-ipfs should track when was the last time content path starting with /ipns/{libp2p-key} was resolved by namesys (ipfs name resolve or ipfs resolve)
and unsubscribe from it after N hours since last lookup.
make it configurable, details TBD:
- there should be an optional config flag with an implicit default (OptionalDuration), so we can adjust the default without impacting user's configs.
- we could just name it Ipns.ReproviderDuration if there is no point in making this ipns-specific – I believe we want to enable reproviding on DHT as well, not sure if there is any value in having separate Ipns.DHTReproviderDuration and Ipns.PubsubReproviderDuration

@schomatis' notes

# IPNS Pubsub Reprovider Duration (GC for unused topics) https://github.com/ipfs/go-ipfs/issues/8586 Related spec: https://github.com/ipfs/specs/blob/master/naming/pubsub.md. Most of the logic of interest (subscription handling) is in the `PubsubValueStore` structure in https://github.com/libp2p/go-libp2p-pubsub-router. Also in the `IpnsResolver` in `github.com/ipfs/go-namesys@v0.4.0/routing.go`. ## Proposed solution My biggest doubt is if we leave the subscription standing because we want to avoid unsubscribing and resubscribing repeatedly or is this just a technical debt. If we actually want to leave the subscription standing then Lidel's proposal seems fine to me and I can add an independent goroutine in `NewPubsubValueStore` (similar to `go psValueStore.rebroadcast(ctx)` [here](https://github.com/libp2p/go-libp2p-pubsub-router/blob/1d5fc5c4ed11ec50a2ce08712f1fd6eb7e108130/pubsub.go#L108)) that periodically checks the last read/write (`PutValue`/`GetValue`) of each topic and unsubscribes after certain time has elapsed (and no one is watching). If, on the other hand, the standing subscriptions are a technical debt then we should have a finer-grained control where we unsubscribe after each topic access (again, if no one is watching it). ## Simplest path to trigger a topic subscription for IPNS using the `ipfs name` commands ``` ipfs config --json Pubsub.Enabled true ipfs config --json Ipns.UsePubsub true # Not working. ipfs daemon --enable-namesys-pubsub --enable-pubsub-experiment & # Do we need both? ipfs name pubsub state # enabled # WARN: We will use the IPNS resolver ONLY for peer IDs arguments, not for any # normal domain that will go through the DNS resolver (we have multiple # resolvers through the `namesys.mpns` abstraction layer). ipfs name resolve --nocache QmWdQmLNaG5rC3kLxJNjYU3E9a6tZ1nhM8kinAZenY8q3c # (Random peer ID extracted from `ipfs swarm peers`) # This will normally just hang (even without the `--stream` flag) trying to # resolve. In the meanwhile we will have generated a subscription that can # be examined through the `ipfs name pubsub` command. (We can even quickly # cancel the command and the subscription will still remain there.) ipfs name pubsub subs --ipns-base v0 # /ipns/QmWdQmLNaG5rC3kLxJNjYU3E9a6tZ1nhM8kinAZenY8q3c # Same peer ID with the `/ipns/` prefix. The encoding argument # `--ipns-base` is very important to not get some weird `base36` # encoding that will obscure this relation. # To cancel (remove the topic from the PubSub vault): ipfs name pubsub cancel /ipns/QmWdQmLNaG5rC3kLxJNjYU3E9a6tZ1nhM8kinAZenY8q3c # If the command is running (even without the `--stream` option) the operation # will fail with `Error: key has active subscriptions` as the # `PubsubValueStore.watching` still has the resolve command registered. # FIXME: Correct description above: we always go through the `SearchValue` API # which will first look for a local copy and if none exist "watch" for the IPNS # topic. # Not sure how the --stream and --nocache options of the `resolve` command map # to this (but not important to look into). ```

lidel commented 2 years ago

cc @BigLep @aschmahmann @Stebalien if the above sounds sensible and something @schomatis could add to his list of things to tackle.

schomatis commented 2 years ago

I have no experience with IPNS in go-ipfs but I think I can figure out the details. Still I'll need support on the earlier stages of implementation (clarifying objectives, providing feedback on the IPNS system, or explaining some details of its implementation here). If there is no one that can provide that help at the moment then probably I shouldn't be assigned.

schomatis commented 2 years ago

Update 12/10 (@schomatis): I think I can tackle this but still will need support during the review process and resolving some doubts. Left my notes of how to proceed at the end of the issue's description. (cc @lidel)

lidel commented 2 years ago

ipfs config --json Ipns.UsePubsub true # Not working.

@schomatis are you using go-ipfs 0.11.0? We've just added those config flags there.

My biggest doubt is if we leave the [IPNS topic] subscription standing because we want to avoid unsubscribing and resubscribing repeatedly or is this just a technical debt.

@aschmahmann mind confirming this? My understanding is that:

Regular pubsub will unsubscribe automatically when the last listener disconnects.
Standing subscriptions are not a bug, but a feature for IPNS over Pubsub, ensuring we get near-realtime updates without the overhead of calling fetch after joining topic over and over again.

schomatis commented 2 years ago

are you using go-ipfs 0.11.0? We've just added those config flags there.

@lidel This is related to https://github.com/ipfs/go-ipfs/issues/7482 but we can ignore it here; I'm setting the options through the daemon's arguments for now.

TobiaszCudnik commented 2 years ago

I took a dab at this to practice my go-fu, and came up with a PR for go-libp2p-pubsub-router which covers the minimal implementation (to my understanding). No tests and no config so far. @schomatis' notes were very useful.

Assumptions

lifetime of IPNS in DHT separate from lifetime in pubsub
DefaultSubscriptionLifetime (pubsub) higher than DefaultRecordEOL (namesys)

Implantation details in the PR itself. Sorry if I DUPed anyones work, feel free to take over anytime.

schomatis commented 2 years ago

Thanks @TobiaszCudnik !! No duplication at all, I haven't started on this. Once the direction above is confirmed we can move forward with this and I'll help you get that landed (or if you're busy I can take over if any finishing touches are required).

BigLep commented 2 years ago

@schomatis : are you able to drive this one to completion with @TobiaszCudnik ?

@lidel : any concerns with @schomatis being the approver for https://github.com/ipfs/go-ipfs/pull/8693 https://github.com/ipfs/go-ipfs-config/pull/165 https://github.com/libp2p/go-libp2p-pubsub-router/pull/92

?

schomatis commented 2 years ago

@BigLep No. We still have open questions about the direction of this issue in https://github.com/ipfs/go-ipfs/issues/8586#issuecomment-992415522, waiting for @aschmahmann 's input.

ipfs / kubo