probe-lab / network-measurements

MIT License
50 stars 13 forks source link

RFM Proposal: Data on usage of libp2p circuit relay v1 #20

Open SgtPooki opened 1 year ago

SgtPooki commented 1 year ago

In https://github.com/ipfs/interop, we still have tests running libp2p circuit relay v1, which makes sense because it has functionality that relay v2 does not; however, It has caused some issues. See

I'm wondering if we can get metrics on which relay versions are being used and how much traffic exists for each. I understand that we should be able to query the DHT for multiaddr that indicate which relay version(s) are available.

As far as which metrics would be useful, I think the following is a good start:

  1. p99/90/50 of relay enablement:
    • is relay enabled for a node?
    • Which versions are they supporting?
  2. relay usage:
    • For relay traffic on the network (X), how much is v1(Y), how much is v2(Z), how much is vN? (X/Y vs X/Z vs X/N...)

Questions I want to answer with this data:

  1. Do we need to support relayv1 in interop tests?
    • Can we remove tests for relayV1 in ipfs/interop? If it's not in use anywhere, yes. If its use is under some percentage when compared to relayV2, then probably..

Please let me know if this request/issue is better suited elsewhere! Thanks.

mxinden commented 1 year ago

we still have tests running libp2p circuit relay v1, which makes sense because it has functionality that relay v2 does not

What is the functionality that v1 has but v2 doesn't @SgtPooki?


If I am not mistaken @dennis-tra is tracking the supported protocols with https://github.com/dennis-tra/nebula-crawler on the IPFS DHT.


Note that rust-libp2p dropped support for circuit relay v1 entirely, see https://github.com/libp2p/rust-libp2p/pull/2549.

dennis-tra commented 1 year ago

Interesting questions, @SgtPooki !

If I am not mistaken @dennis-tra is tracking the supported protocols with https://github.com/dennis-tra/nebula-crawler on the IPFS DHT.

Just checked the latest crawl and these are the numbers (out of 11,885 total online peers):

image

The numbers on the right are the number of peers that reported to support the protocols on the left.

I'm wondering if we can get metrics on which relay versions are being used and how much traffic exists for each.

The above numbers indicate which relay versions are supported - not necessarily which and how much they are used. I believe this is hard to come by with external observations only 🤔 However, do the numbers already give a hint to your p99/90/50 of relay enablement questions?

  1. p99/90/50 of relay enablement:
    • is relay enabled for a node?
    • Which versions are they supporting?

relay usage: For relay traffic on the network (X), how much is v1(Y), how much is v2(Z), how much is vN? (X/Y vs X/Z vs X/N...)

Just thinking out loud here: We could extrapolate measurements of PL controlled relay nodes. We could measure the relayed traffic with either protocol (v1/v2) and extrapolate to the whole network based on the above numbers? This methodology assumes that relayed traffic is evenly distributed among all relay nodes.

SgtPooki commented 1 year ago

Thanks a lot @dennis-tra! Those numbers make it seem like relay V1 is vital: 100% of nodes rounding to the nearest thousandth have it enabled. However, without knowing how many of those nodes are relaying any traffic, let alone v1 or v2, our support efforts for v1 are still flying blind.

Extrapolating some numbers we could compare would be great for setting ourselves up for sunsetting v1/v2/v3... etc.. in the future.

FYI the issues mentioned in the issue description have since been resolved.

SgtPooki commented 1 year ago

What is the functionality that v1 has but v2 doesn't @SgtPooki?

I am not sure Max.. I believe it was something lidel mentioned.

SgtPooki commented 1 year ago

@lidel says you cannot send data over v2 (no bitswap). I think the key here is that understanding how much traffic, over what protocols, is vital for understanding what priorities and efforts we should have.

bcc @TheDiscordian

mxinden commented 1 year ago

@lidel says you cannot send data over v2 (no bitswap). I think the key here is that understanding how much traffic, over what protocols, is vital for understanding what priorities and efforts we should have.

I am assuming that you are referring to the fact that Circuit Relay v2 allows a relay node to enforce a limit on the amount of data to be relayed, right?

Note that that is a feature, not a restriction. In other words, a Circuit Relay v2 relay node can offer the same functionality like a Circuit Relay v1 node if one sets these limits to a high value.

All recent public IPFS nodes act as a limited Circuit Relay v2 node. But that doesn't stop folks from running an unlimited Circuit Relay v2 node for e.g. a bitswap use-case.

BigLep commented 1 year ago

@SgtPooki: given your original concern is mostly about improving the stability of ipfs/interop because of issues with relay-v1, I think the key thing we need to push on relay-v2 support in js: https://github.com/libp2p/js-libp2p/issues/1029 . Landing that helps the story f getting rid of v1 in ipfs/interop. I agree that data of relay-v1 usage in production would be useful, but there is also a degree of its functionality we don't want to support on policy which can lead us to just dropping it as rust-libp2p has done.

As a result, I think this specific RFM can likely be closed.

SgtPooki commented 1 year ago

@BigLep maybe this doesn't need prioritized over other things, but I think closing this is unwise.

@mxinden https://docs.ipfs.tech/concepts/nodes/#relay also indicates that relay v2 are "limited relays":

v2 relays are "limited relays" that are designed to be used for Direct Connection Upgrade through Relay (opens new window)(aka hole punching).

Maybe a temporary relay would be a better name, but "limited relay" implies that relayv1 does things that v2 does not