send tracing output to context-provided sink

[raulk]

A --trace flag in DHT diagnostics tools like ipfs dht and Filecoin's would cause the DHT to emit logs twofold, by tee'ing it into both its original sink, and a context-specified one. The caller wires the latter into its frontend.

This allows the DHT to "narrate" how it's resolving the user's query.

In the context of the aforementioned tools, since their tooling works over HTTP, the context-provided sink would wire into the http.ResponseWriter, or whichever abstraction they're using in their commands handling library, which would ultimately stream the tracing output from the libp2p host to the CLI tool.

Example:

// in the DHT, export this struct for the key value.
type TracingSink struct{}

// in the caller
// (e.g. ipfs dht command handler: https://github.com/ipfs/go-ipfs/blob/master/core/commands/dht.go#L380)

// sink adapts cmd.ResponseEmitter to the log.StandardLogger interface.
var sink log.StandardLogger
ctx := context.WithValue(context.WithTimeout(context.Background(), 10*time.Second), dht.TracingSink{}, sink))
p, err := dht.FindPeer(ctx, "target")

[raulk]

@anacrolix – we had discussed this solution 1:1 but never registered it. Are you still onboard with this, or has your thinking changed?

[raulk]

Relates to #256.

[anacrolix]

I still think it's an interesting idea. I wonder how it overlaps with concepts of tracing floating around in some metrics packages.

[anacrolix]

This is easy enough to implement, but requires more information on how and what consumers need.

[ZenGround0]

FYI the existing go-ipfs dht command (and the go-filecoin dht command copied mostly from ipfs) use a similar pattern by directly reading and writing to the libp2p event system using go-libp2p-routing/notifications. See here.

It is unclear if this serves part of the intended purpose of the tracing tracked here to me. It does seem similar though maybe you are all well aware of this feature. I was prompted to post about this here by @anacrolix.

Filecoin does not yet have a good read on our exact needs from debug traces. What is described here SGTM but I don't have much more specific guidance on what filecoin needs. @anacrolix we could sync up on intended uses if consumer input is blocking you. However, I believe the features that you deem useful from the perspective of the dht analysis tool (that I believe will also use this feature) are likely general enough to be good for us too.

[lanzafame]

@raulk would something like OC zpages fit the bill: https://opencensus.io/zpages/go/