jessicadaugherty
commented
2 years ago @andrewnguyen22 @deblasis for your review
Closed jessicadaugherty closed 1 year ago
jessicadaugherty
commented
2 years ago @andrewnguyen22 @deblasis for your review
Olshansk
commented
2 years ago @deblasis I added a new paper in https://github.com/pokt-network/pocket/pull/305 that talks about eclipse attacks in Geth and it's a great starting point for understanding: https://arxiv.org/pdf/1908.10141.pdf.
@jessicadaugherty I would say that there are 4 things we can take from LibP2P:
Even if we don't use (1) and (2), we could still potentially leverage (3) & (4). That should be part of the scope of work.
Olshansk
commented
2 years ago Updated the description above.
jessicadaugherty
commented
2 years ago Thanks @Olshansk!
Olshansk
commented
2 years ago @deblasis In addition to presenting the research to the team at a protocol hour at some point, could you also post aa "short" version of your research here once its done?
I was thinking of something similar to the SMT evaluation here: https://github.com/pokt-network/pocket/issues/199#issuecomment-1309335405.
deblasis
commented
2 years ago I have spent some time "playing with it", exploring the source code and reading the available documentation, some of the issues and PRs.
"When something seems too good to be true, it probably is" - Someone smarter than me
This thought has been bugging me quite a bit and I didn't want to commit to something that would have shown its limitations pretty quickly.
The library appears:
Out-of-the-box, perhaps it doesn't offer what we are looking for. It requires some adaptations and I was looking specifically at how to minimize the potential blast radius of the changes, so that keeping the codebase aligned with upstream would be as painless as possible.
The default implementation for DHT is Kademlia based with improvements and oriented towards content discovery and it's not really optimized for the type of message propagation and redundancy we are looking for with Raintree in mind.
Basically, it has been developed with in mind the requirement of being able to PUT and GET keys into/from close nodes so that it's possible to understand which nodes provide the required key which is a surrogate for some content, like a file for example (IPFS).
Our use case is simpler:
Luckily, the library is pretty modular and peer discovery and peer routing could be implemented in a custom way (Raintree or Gemini - or any other implementation, I guess - come to mind really).
They used some of the patterns that we use already in our codebase:
Options that are used to attach behaviour/configurationI think it shouldn't be too much of a learning curve and we can actually contribute back as we might eventually bump into unforeseen limitations/bugs.
The library provides interfaces that can be implemented to Advertise and FindPeers. Channels are used as a primitive to convey the changes. This allows the library user to develop their own implementation or to use one of the ones offered (there are a few examples: rendezvouz and multicast DNS)
@Olshansk this is also to say that perhaps you gathered incorrectly that the library "provides peer discovery" as you hinted at in one of our conversations.
Yes/No, really. It does for the default routing which is Kademlia DHT.
Substantially, if you take the core library there's no logic for it, only the interface
// PeerRouting is a way to find address information about certain peers.
// This can be implemented by a simple lookup table, a tracking server,
// or even a DHT.
type PeerRouting interface {
// FindPeer searches for a peer with given ID, returns a peer.AddrInfo
// with relevant addresses.
FindPeer(context.Context, peer.ID) (peer.AddrInfo, error)
}The routing table is an implementation detail that depends on the DHT, therefore it has to be developed accordingly, as a starting point, we have the default implementations that have to be altered depending on our algorithm(s) of choice.
Kademlia DHT uses https://github.com/libp2p/go-libp2p-kbucket as routing table.
github.com/libp2p/go-libp2p-core/network.Network takes care of maintaining streams, connections, multiplexing different protocols on them, handling incoming connections etc.
The core library implements also a ConnManager
// ConnManager tracks connections to peers, and allows consumers to associate
// metadata with each peer.
//
// It enables connections to be trimmed based on implementation-defined
// heuristics. The ConnManager allows libp2p to enforce an upper bound on the
// total number of open connections.
//
// ConnManagers supporting decaying tags implement Decayer. Use the
// SupportsDecay function to safely cast an instance to Decayer, if supported.and also introspection, so that every peer can tell (internally or externally) how many sessions are currently open and with which other peers. This could be handy for troubleshooting.
Security is implemented, again, as a plug-in:
// Security configures libp2p to use the given security transport (or transport
// constructor).
//
// Name is the protocol name.
//
// The transport can be a constructed security.Transport or a function taking
// any subset of this libp2p node's:
// * Public key
// * Private key
// * Peer ID
// * Host
// * Network
// * PeerstoreTLS, specifically is quite trivial to implement/use out-of-the-box.
I would like the team to be aligned on our next steps. Especially if there are concerns of any kind. We could also use libp2p just as a framework, providing primitives, domain specific patterns, nomenclature, etc.
I was putting together a presentation that also included some Gemini thoughts, given the circumstances (Gemini being descoped), I realise that perhaps a better use of everyone's time is if I can ask you guys to have a read ad this 👆 and come up with some questions that I will gladly focus on specifically during a protocol hour.
Everything else, I guess we can start adding some tasks to cover the integration points and it looks like you guys are already on it. 🚀
deblasis
commented
2 years ago @Olshansk:
@deblasis I added a new paper in pokt-network/pocket#305 that talks about eclipse attacks in Geth and it's a great starting point for understanding: https://arxiv.org/pdf/1908.10141.pdf.
The language might be not appropriate, I'd package it differently in a document, this is mostly for sharing my thoughts in a, hopefully, digestible way with lots of metaphors and a joke here and there.
As usual, brutally honest feedback welcome and encouragedFrom my rough notes:
PeerIDs sounds like a good idea but anything that's reducible to bits is, one way or another, forgeable and therefore its properties can be exploited to induce a certain behaviour in an honest peer from an attacker.These things are very much dependent on the implementation details of the algorithms that we are going to use/develop.
From my readings, I spotted a common pattern:
The routing table is used in such a way that's possible to determine a measure of "distance" or closeness between peers. This is achieved either by using bitwise XOR (Kademlia and geth) or by hashing parts of the address to define groups (hatgroups) of peers like in the case of Gemini for example, or other similar approaches.
What comes into play as a potential attack vector is all the orchestration related to peer discovery and churn. We can learn mitigation strategies from that paper and previous work, specifically I'd quote Heilman's et al Bitcoin related work on eclipse attacks https://www.usenix.org/system/files/conference/usenixsecurity15/sec15-paper-heilman.pdf
Trust is often reliant on some form of identification, especially in the domain of network communication.
I can always say I am somebody but I could be wearing a mask and/or having forged Ids.
If we simply trust a peer because it says "hi" to us (ping) that's a huge flaw.
You wouldn't give the keys of your house to a perfect stranger just because they are polite.
If we evict good peers from our k-buckets just because a supposedly new and "fresher" peer pings us, given time, our buckets can be filled with malicious peers... (this is an example of eclipse attack)
Cryptographic signature verification aims at solving the identification problem but the fact that we might have malicious (byzantine) actors in the network is often "forgotten naively" or maybe it's just a very hard problem to solve.
When derivatives of private keys like the hash of the public key are used to determine closeness between peers, the assumption is that all peers are good actors. In my opinion we should be defensive in that sense. Trust has to be earned somehow. Either via PoS or some other "unconfutable" way (ZK-Stuff™️ - credit to @Olshansk for the name) IMHO.
There's no free lunch and perhaps additional security could involve more network hops (ideally O(n)), more coordination between peers (more complexity) or any other unforeseen tradeoff. Is it worth it though? IMHO, yes, security should be a first-class citizen especially if we are building something from scratch.
I'd probably consider bounties for white-hat/ethical hackers at some point, after OSS has done its magic with more altruistic personas.
These are probably conversations to be held at some point when our P2P stack matures, with a PoC in our calendar, this is probably out-of-scope for now (we are considering happy paths as a starting point) but we need to convey the message, loud and clear, that we are navigating almost uncharted waters, we only know that they are infested by sharks and that there are pirates and sirens. This is out of intellectual honesty and for setting the expectations.
To summarise: it's not going to be an easy feat to pull out but I think that the team can do it.
My hunch is that we should embed some trust in the identities of the peers that would make it harder aka expensive for attackers to either eclipse or simply DoS a peer.
Maybe a PeerId that embeds a proof of stake of some sort that can be used along timestamps to sort peers within k-buckets. 🤔
It's the "Twitter verified checkmark" for peers basically, but there's no 8$ fee and it's on-chain (no Elon 😉 ).
Before evicting a peer from a bucket because a fresher one pings us, we could run a liveness check. And potentially ignore the new peer or adding it but with an older timestamp (basically if the liveness check of an "old friend" is positive, it would be considered "more alive" than the "new guy in the block".
when selecting a random peer, we should consider the widest possible k-bucket, ideally containing the whole addressBook but that's probably unrealistic.
Olshansk
commented
2 years ago @deblasis Appreciate the notes and research, in particular the jokes are 💯. My immediative feedback is that the narrative style covers a lot but organizing it more into topics and splitting things by problems/solutions or short/long term would have made it easier to understand in what direction the ideas are heading. As discussed offline, getting more feedback on the technicals (networking layer, codebase, projects that use libp2p, testing/code practices, etc...) would have been really helpful.
I wanted to summarize next steps in terms of research to close this out, we need to answer these questions:
deblasis
commented
2 years ago 🙏I appreciate the feedback @Olshansk, the truth is that I wasn't ready yet to present my findings, I was preparing a presentation that also partially covered Gemini using it as an example to swap the routing algorithm in LibP2P but then we moved on from that. I rushed into summarizing a short version of my findings and that contributed to the fact that my exposition was particularly messy :)
Regardless, gathering feedback from you and the team was beneficial. I didn't know exactly about the pains in V0 with maintaining dependencies otherwise I would have focused more on "quality metrics" instead of simply saying "LGTM" and focusing on the points mentioned in this issue description (peer discovery and management, etc).
This is a team decision, therefore I am very happy to dig deeper for the team's and my own benefit.
For the records, my analysis has been performed on commit d8d2efaf
Source: go-libp2p-stats.zip
1. Quality: What is LibP2P current quality control process (test coverage, testing procedure, etc...)? Note that this can be an endless task so a high level approach is sufficient.
TL;DR, IMHO very good ✅
I sampled the lower values and it appears that they are mostly in mocks and files that have many interfaces and maybe just one function/method that has no tests.
| File | Coverage % |
|---|---|
| github.com/libp2p/go-libp2p/config/config.go | 72.5% |
| github.com/libp2p/go-libp2p/config/log.go | 100.0% |
| github.com/libp2p/go-libp2p/config/muxer.go | 88.9% |
| github.com/libp2p/go-libp2p/config/security.go | 57.1% |
| github.com/libp2p/go-libp2p/core/canonicallog/canonicallog.go | 84.6% |
| github.com/libp2p/go-libp2p/core/connmgr/manager.go | 100.0% |
| github.com/libp2p/go-libp2p/core/connmgr/null.go | 40.0% |
| github.com/libp2p/go-libp2p/core/connmgr/presets.go | 90.9% |
| github.com/libp2p/go-libp2p/core/crypto/ecdsa.go | 74.5% |
| github.com/libp2p/go-libp2p/core/crypto/ed25519.go | 95.1% |
| github.com/libp2p/go-libp2p/core/crypto/key.go | 74.1% |
| github.com/libp2p/go-libp2p/core/crypto/key_not_openssl.go | 78.6% |
| github.com/libp2p/go-libp2p/core/crypto/pb/crypto.pb.go | 64.1% |
| github.com/libp2p/go-libp2p/core/crypto/rsa_common.go | 66.7% |
| github.com/libp2p/go-libp2p/core/crypto/rsa_go.go | 93.2% |
| github.com/libp2p/go-libp2p/core/crypto/secp256k1.go | 93.6% |
| github.com/libp2p/go-libp2p/core/discovery/options.go | 90.0% |
| github.com/libp2p/go-libp2p/core/host/helpers.go | 100.0% |
| github.com/libp2p/go-libp2p/core/internal/catch/catch.go | 100.0% |
| github.com/libp2p/go-libp2p/core/introspection/pb/introspection.pb.go | 0.7% |
| github.com/libp2p/go-libp2p/core/metrics/bandwidth.go | 93.1% |
| github.com/libp2p/go-libp2p/core/network/context.go | 100.0% |
| github.com/libp2p/go-libp2p/core/network/errors.go | 33.3% |
| github.com/libp2p/go-libp2p/core/network/mocks/mock_conn_management_scope.go | 46.5% |
| github.com/libp2p/go-libp2p/core/network/mocks/mock_peer_scope.go | 0.0% |
| github.com/libp2p/go-libp2p/core/network/mocks/mock_protocol_scope.go | 0.0% |
| github.com/libp2p/go-libp2p/core/network/mocks/mock_resource_manager.go | 33.3% |
| github.com/libp2p/go-libp2p/core/network/mocks/mock_stream_management_scope.go | 23.0% |
| github.com/libp2p/go-libp2p/core/network/nattype.go | 0.0% |
| github.com/libp2p/go-libp2p/core/network/network.go | 0.0% |
| github.com/libp2p/go-libp2p/core/network/notifee.go | 100.0% |
| github.com/libp2p/go-libp2p/core/network/rcmgr.go | 50.0% |
| github.com/libp2p/go-libp2p/core/peer/addrinfo.go | 73.9% |
| github.com/libp2p/go-libp2p/core/peer/addrinfo_serde.go | 90.0% |
| github.com/libp2p/go-libp2p/core/peer/pb/peer_record.pb.go | 52.5% |
| github.com/libp2p/go-libp2p/core/peer/peer.go | 73.8% |
| github.com/libp2p/go-libp2p/core/peer/peer_serde.go | 78.9% |
| github.com/libp2p/go-libp2p/core/peer/record.go | 81.2% |
| github.com/libp2p/go-libp2p/core/peerstore/helpers.go | 0.0% |
| github.com/libp2p/go-libp2p/core/peerstore/peerstore.go | 100.0% |
| github.com/libp2p/go-libp2p/core/pnet/codec.go | 76.9% |
| github.com/libp2p/go-libp2p/core/pnet/env.go | 100.0% |
| github.com/libp2p/go-libp2p/core/pnet/error.go | 100.0% |
| github.com/libp2p/go-libp2p/core/protocol/id.go | 100.0% |
| github.com/libp2p/go-libp2p/core/record/envelope.go | 86.6% |
| github.com/libp2p/go-libp2p/core/record/pb/envelope.pb.go | 64.5% |
| github.com/libp2p/go-libp2p/core/record/record.go | 100.0% |
| github.com/libp2p/go-libp2p/core/routing/options.go | 0.0% |
| github.com/libp2p/go-libp2p/core/routing/query.go | 90.5% |
| github.com/libp2p/go-libp2p/core/routing/query_serde.go | 0.0% |
| github.com/libp2p/go-libp2p/core/routing/routing.go | 0.0% |
| github.com/libp2p/go-libp2p/core/sec/insecure/insecure.go | 82.6% |
| github.com/libp2p/go-libp2p/core/sec/insecure/pb/plaintext.pb.go | 48.3% |
| github.com/libp2p/go-libp2p/core/test/addrs.go | 33.3% |
| github.com/libp2p/go-libp2p/core/test/crypto.go | 100.0% |
| github.com/libp2p/go-libp2p/core/test/errors.go | 66.7% |
| github.com/libp2p/go-libp2p/core/test/peer.go | 87.5% |
| github.com/libp2p/go-libp2p/defaults.go | 75.5% |
| github.com/libp2p/go-libp2p/libp2p.go | 91.7% |
| github.com/libp2p/go-libp2p/limits.go | 100.0% |
| github.com/libp2p/go-libp2p/options.go | 69.2% |
| github.com/libp2p/go-libp2p/options_filter.go | 0.0% |
| github.com/libp2p/go-libp2p/p2p/discovery/backoff/backoff.go | 69.8% |
| github.com/libp2p/go-libp2p/p2p/discovery/backoff/backoffcache.go | 79.9% |
| github.com/libp2p/go-libp2p/p2p/discovery/backoff/backoffconnector.go | 90.9% |
| github.com/libp2p/go-libp2p/p2p/discovery/mdns/mdns.go | 80.2% |
| github.com/libp2p/go-libp2p/p2p/discovery/mocks/mocks.go | 83.7% |
| github.com/libp2p/go-libp2p/p2p/discovery/routing/routing.go | 80.0% |
| github.com/libp2p/go-libp2p/p2p/discovery/util/util.go | 30.0% |
| github.com/libp2p/go-libp2p/p2p/host/autonat/autonat.go | 83.7% |
| github.com/libp2p/go-libp2p/p2p/host/autonat/client.go | 66.7% |
| github.com/libp2p/go-libp2p/p2p/host/autonat/dialpolicy.go | 86.8% |
| github.com/libp2p/go-libp2p/p2p/host/autonat/notify.go | 100.0% |
| github.com/libp2p/go-libp2p/p2p/host/autonat/options.go | 70.5% |
| github.com/libp2p/go-libp2p/p2p/host/autonat/pb/autonat.pb.go | 56.5% |
| github.com/libp2p/go-libp2p/p2p/host/autonat/proto.go | 100.0% |
| github.com/libp2p/go-libp2p/p2p/host/autonat/svc.go | 74.8% |
| github.com/libp2p/go-libp2p/p2p/host/autorelay/addrsplosion.go | 95.7% |
| github.com/libp2p/go-libp2p/p2p/host/autorelay/autorelay.go | 85.7% |
| github.com/libp2p/go-libp2p/p2p/host/autorelay/host.go | 100.0% |
| github.com/libp2p/go-libp2p/p2p/host/autorelay/options.go | 87.2% |
| github.com/libp2p/go-libp2p/p2p/host/autorelay/relay.go | 100.0% |
| github.com/libp2p/go-libp2p/p2p/host/autorelay/relay_finder.go | 87.0% |
| github.com/libp2p/go-libp2p/p2p/host/autorelay/timer.go | 75.0% |
| github.com/libp2p/go-libp2p/p2p/host/basic/basic_host.go | 69.5% |
| github.com/libp2p/go-libp2p/p2p/host/basic/natmgr.go | 25.3% |
| github.com/libp2p/go-libp2p/p2p/host/basic/peer_connectedness.go | 95.0% |
| github.com/libp2p/go-libp2p/p2p/host/blank/blank.go | 70.0% |
| github.com/libp2p/go-libp2p/p2p/host/blank/peer_connectedness.go | 95.0% |
| github.com/libp2p/go-libp2p/p2p/host/eventbus/basic.go | 94.8% |
| github.com/libp2p/go-libp2p/p2p/host/eventbus/opts.go | 100.0% |
| github.com/libp2p/go-libp2p/p2p/host/peerstore/metrics.go | 94.7% |
| github.com/libp2p/go-libp2p/p2p/host/peerstore/pb/custom.go | 28.6% |
| github.com/libp2p/go-libp2p/p2p/host/peerstore/pb/pstore.pb.go | 50.3% |
| github.com/libp2p/go-libp2p/p2p/host/peerstore/peerstore.go | 0.0% |
| github.com/libp2p/go-libp2p/p2p/host/peerstore/pstoreds/addr_book.go | 87.1% |
| github.com/libp2p/go-libp2p/p2p/host/peerstore/pstoreds/addr_book_gc.go | 69.7% |
| github.com/libp2p/go-libp2p/p2p/host/peerstore/pstoreds/cache.go | 25.0% |
| github.com/libp2p/go-libp2p/p2p/host/peerstore/pstoreds/cyclic_batch.go | 67.9% |
| github.com/libp2p/go-libp2p/p2p/host/peerstore/pstoreds/keybook.go | 60.3% |
| github.com/libp2p/go-libp2p/p2p/host/peerstore/pstoreds/metadata.go | 82.6% |
| github.com/libp2p/go-libp2p/p2p/host/peerstore/pstoreds/peerstore.go | 84.2% |
| github.com/libp2p/go-libp2p/p2p/host/peerstore/pstoreds/protobook.go | 89.2% |
| github.com/libp2p/go-libp2p/p2p/host/peerstore/pstoremem/addr_book.go | 91.4% |
| github.com/libp2p/go-libp2p/p2p/host/peerstore/pstoremem/keybook.go | 92.9% |
| github.com/libp2p/go-libp2p/p2p/host/peerstore/pstoremem/metadata.go | 95.2% |
| github.com/libp2p/go-libp2p/p2p/host/peerstore/pstoremem/peerstore.go | 90.0% |
| github.com/libp2p/go-libp2p/p2p/host/peerstore/pstoremem/protobook.go | 97.5% |
| github.com/libp2p/go-libp2p/p2p/host/peerstore/pstoremem/sorting.go | 94.4% |
| github.com/libp2p/go-libp2p/p2p/host/peerstore/test/addr_book_suite.go | 90.6% |
| github.com/libp2p/go-libp2p/p2p/host/peerstore/test/benchmarks_suite.go | 0.0% |
| github.com/libp2p/go-libp2p/p2p/host/peerstore/test/keybook_suite.go | 37.6% |
| github.com/libp2p/go-libp2p/p2p/host/peerstore/test/peerstore_suite.go | 90.1% |
| github.com/libp2p/go-libp2p/p2p/host/peerstore/test/utils.go | 54.1% |
| github.com/libp2p/go-libp2p/p2p/host/pstoremanager/pstoremanager.go | 85.4% |
| github.com/libp2p/go-libp2p/p2p/host/relaysvc/relay.go | 75.7% |
| github.com/libp2p/go-libp2p/p2p/host/resource-manager/allowlist.go | 88.8% |
| github.com/libp2p/go-libp2p/p2p/host/resource-manager/error.go | 93.3% |
| github.com/libp2p/go-libp2p/p2p/host/resource-manager/extapi.go | 0.0% |
| github.com/libp2p/go-libp2p/p2p/host/resource-manager/limit.go | 72.5% |
| github.com/libp2p/go-libp2p/p2p/host/resource-manager/limit_defaults.go | 83.9% |
| github.com/libp2p/go-libp2p/p2p/host/resource-manager/metrics.go | 57.8% |
| github.com/libp2p/go-libp2p/p2p/host/resource-manager/obs/stats.go | 16.7% |
| github.com/libp2p/go-libp2p/p2p/host/resource-manager/rcmgr.go | 74.6% |
| github.com/libp2p/go-libp2p/p2p/host/resource-manager/scope.go | 89.8% |
| github.com/libp2p/go-libp2p/p2p/host/resource-manager/sys_unix.go | 60.0% |
| github.com/libp2p/go-libp2p/p2p/host/resource-manager/trace.go | 30.8% |
| github.com/libp2p/go-libp2p/p2p/host/routed/routed.go | 41.4% |
| github.com/libp2p/go-libp2p/p2p/muxer/mplex/conn.go | 83.3% |
| github.com/libp2p/go-libp2p/p2p/muxer/mplex/stream.go | 75.0% |
| github.com/libp2p/go-libp2p/p2p/muxer/mplex/transport.go | 75.0% |
| github.com/libp2p/go-libp2p/p2p/muxer/muxer-multistream/multistream.go | 92.3% |
| github.com/libp2p/go-libp2p/p2p/muxer/testsuite/mux.go | 85.8% |
| github.com/libp2p/go-libp2p/p2p/muxer/yamux/conn.go | 100.0% |
| github.com/libp2p/go-libp2p/p2p/muxer/yamux/stream.go | 100.0% |
| github.com/libp2p/go-libp2p/p2p/muxer/yamux/transport.go | 94.7% |
| github.com/libp2p/go-libp2p/p2p/net/conn-security-multistream/ssms.go | 81.4% |
| github.com/libp2p/go-libp2p/p2p/net/conngater/conngater.go | 78.4% |
| github.com/libp2p/go-libp2p/p2p/net/connmgr/connmgr.go | 74.6% |
| github.com/libp2p/go-libp2p/p2p/net/connmgr/decay.go | 91.7% |
| github.com/libp2p/go-libp2p/p2p/net/connmgr/options.go | 87.5% |
| github.com/libp2p/go-libp2p/p2p/net/connmgr/watchdog_cgo.go | 0.0% |
| github.com/libp2p/go-libp2p/p2p/net/mock/complement.go | 50.0% |
| github.com/libp2p/go-libp2p/p2p/net/mock/interface.go | 0.0% |
| github.com/libp2p/go-libp2p/p2p/net/mock/mock.go | 70.6% |
| github.com/libp2p/go-libp2p/p2p/net/mock/mock_conn.go | 89.2% |
| github.com/libp2p/go-libp2p/p2p/net/mock/mock_link.go | 82.9% |
| github.com/libp2p/go-libp2p/p2p/net/mock/mock_net.go | 79.9% |
| github.com/libp2p/go-libp2p/p2p/net/mock/mock_peernet.go | 94.7% |
| github.com/libp2p/go-libp2p/p2p/net/mock/mock_printer.go | 0.0% |
| github.com/libp2p/go-libp2p/p2p/net/mock/mock_stream.go | 90.0% |
| github.com/libp2p/go-libp2p/p2p/net/mock/ratelimiter.go | 96.3% |
| github.com/libp2p/go-libp2p/p2p/net/nat/mapping.go | 0.0% |
| github.com/libp2p/go-libp2p/p2p/net/nat/nat.go | 4.3% |
| github.com/libp2p/go-libp2p/p2p/net/pnet/protector.go | 80.0% |
| github.com/libp2p/go-libp2p/p2p/net/pnet/psk_conn.go | 82.1% |
| github.com/libp2p/go-libp2p/p2p/net/reuseport/dial.go | 85.2% |
| github.com/libp2p/go-libp2p/p2p/net/reuseport/dialer.go | 88.2% |
| github.com/libp2p/go-libp2p/p2p/net/reuseport/listen.go | 78.8% |
| github.com/libp2p/go-libp2p/p2p/net/reuseport/reuseport.go | 90.0% |
| github.com/libp2p/go-libp2p/p2p/net/reuseport/reuseport_posix.go | 100.0% |
| github.com/libp2p/go-libp2p/p2p/net/swarm/addrs.go | 75.0% |
| github.com/libp2p/go-libp2p/p2p/net/swarm/dial_error.go | 56.2% |
| github.com/libp2p/go-libp2p/p2p/net/swarm/dial_sync.go | 97.1% |
| github.com/libp2p/go-libp2p/p2p/net/swarm/dial_worker.go | 91.4% |
| github.com/libp2p/go-libp2p/p2p/net/swarm/limiter.go | 97.5% |
| github.com/libp2p/go-libp2p/p2p/net/swarm/swarm.go | 87.2% |
| github.com/libp2p/go-libp2p/p2p/net/swarm/swarm_addr.go | 93.1% |
| github.com/libp2p/go-libp2p/p2p/net/swarm/swarm_conn.go | 86.6% |
| github.com/libp2p/go-libp2p/p2p/net/swarm/swarm_dial.go | 82.2% |
| github.com/libp2p/go-libp2p/p2p/net/swarm/swarm_listen.go | 86.8% |
| github.com/libp2p/go-libp2p/p2p/net/swarm/swarm_stream.go | 91.2% |
| github.com/libp2p/go-libp2p/p2p/net/swarm/swarm_transport.go | 73.1% |
| github.com/libp2p/go-libp2p/p2p/net/swarm/testing/testing.go | 91.1% |
| github.com/libp2p/go-libp2p/p2p/net/upgrader/conn.go | 44.4% |
| github.com/libp2p/go-libp2p/p2p/net/upgrader/listener.go | 86.4% |
| github.com/libp2p/go-libp2p/p2p/net/upgrader/threshold.go | 94.1% |
| github.com/libp2p/go-libp2p/p2p/net/upgrader/upgrader.go | 80.0% |
| github.com/libp2p/go-libp2p/p2p/protocol/circuitv1/pb/circuitv1.pb.go | 54.8% |
| github.com/libp2p/go-libp2p/p2p/protocol/circuitv1/relay/options.go | 14.3% |
| github.com/libp2p/go-libp2p/p2p/protocol/circuitv1/relay/relay.go | 59.7% |
| github.com/libp2p/go-libp2p/p2p/protocol/circuitv2/client/client.go | 100.0% |
| github.com/libp2p/go-libp2p/p2p/protocol/circuitv2/client/conn.go | 70.6% |
| github.com/libp2p/go-libp2p/p2p/protocol/circuitv2/client/dial.go | 66.9% |
| github.com/libp2p/go-libp2p/p2p/protocol/circuitv2/client/handlers.go | 60.2% |
| github.com/libp2p/go-libp2p/p2p/protocol/circuitv2/client/listen.go | 73.3% |
| github.com/libp2p/go-libp2p/p2p/protocol/circuitv2/client/reservation.go | 82.7% |
| github.com/libp2p/go-libp2p/p2p/protocol/circuitv2/client/transport.go | 69.7% |
| github.com/libp2p/go-libp2p/p2p/protocol/circuitv2/pb/circuit.pb.go | 58.4% |
| github.com/libp2p/go-libp2p/p2p/protocol/circuitv2/pb/voucher.pb.go | 52.7% |
| github.com/libp2p/go-libp2p/p2p/protocol/circuitv2/proto/voucher.go | 85.0% |
| github.com/libp2p/go-libp2p/p2p/protocol/circuitv2/relay/constraints.go | 97.9% |
| github.com/libp2p/go-libp2p/p2p/protocol/circuitv2/relay/options.go | 33.3% |
| github.com/libp2p/go-libp2p/p2p/protocol/circuitv2/relay/relay.go | 61.3% |
| github.com/libp2p/go-libp2p/p2p/protocol/circuitv2/relay/resources.go | 100.0% |
| github.com/libp2p/go-libp2p/p2p/protocol/circuitv2/util/io.go | 88.9% |
| github.com/libp2p/go-libp2p/p2p/protocol/circuitv2/util/pbconv.go | 89.5% |
| github.com/libp2p/go-libp2p/p2p/protocol/holepunch/holepuncher.go | 80.2% |
| github.com/libp2p/go-libp2p/p2p/protocol/holepunch/pb/holepunch.pb.go | 47.9% |
| github.com/libp2p/go-libp2p/p2p/protocol/holepunch/svc.go | 69.8% |
| github.com/libp2p/go-libp2p/p2p/protocol/holepunch/tracer.go | 61.4% |
| github.com/libp2p/go-libp2p/p2p/protocol/holepunch/util.go | 87.1% |
| github.com/libp2p/go-libp2p/p2p/protocol/identify/id.go | 82.4% |
| github.com/libp2p/go-libp2p/p2p/protocol/identify/id_delta.go | 68.6% |
| github.com/libp2p/go-libp2p/p2p/protocol/identify/id_go118.go | 23.8% |
| github.com/libp2p/go-libp2p/p2p/protocol/identify/id_push.go | 100.0% |
| github.com/libp2p/go-libp2p/p2p/protocol/identify/obsaddr.go | 79.4% |
| github.com/libp2p/go-libp2p/p2p/protocol/identify/opts.go | 100.0% |
| github.com/libp2p/go-libp2p/p2p/protocol/identify/pb/identify.pb.go | 64.1% |
| github.com/libp2p/go-libp2p/p2p/protocol/identify/peer_loop.go | 93.9% |
| github.com/libp2p/go-libp2p/p2p/protocol/internal/circuitv1-deprecated/conn.go | 42.4% |
| github.com/libp2p/go-libp2p/p2p/protocol/internal/circuitv1-deprecated/dial.go | 81.0% |
| github.com/libp2p/go-libp2p/p2p/protocol/internal/circuitv1-deprecated/listen.go | 62.5% |
| github.com/libp2p/go-libp2p/p2p/protocol/internal/circuitv1-deprecated/pb/relay.pb.go | 54.8% |
| github.com/libp2p/go-libp2p/p2p/protocol/internal/circuitv1-deprecated/relay.go | 22.3% |
| github.com/libp2p/go-libp2p/p2p/protocol/internal/circuitv1-deprecated/transport.go | 66.7% |
| github.com/libp2p/go-libp2p/p2p/protocol/internal/circuitv1-deprecated/util.go | 77.5% |
| github.com/libp2p/go-libp2p/p2p/protocol/ping/ping.go | 68.9% |
| github.com/libp2p/go-libp2p/p2p/security/noise/crypto.go | 100.0% |
| github.com/libp2p/go-libp2p/p2p/security/noise/handshake.go | 86.7% |
| github.com/libp2p/go-libp2p/p2p/security/noise/pb/payload.pb.go | 57.1% |
| github.com/libp2p/go-libp2p/p2p/security/noise/rw.go | 94.3% |
| github.com/libp2p/go-libp2p/p2p/security/noise/session.go | 88.5% |
| github.com/libp2p/go-libp2p/p2p/security/noise/session_transport.go | 94.7% |
| github.com/libp2p/go-libp2p/p2p/security/noise/transport.go | 94.7% |
| github.com/libp2p/go-libp2p/p2p/security/tls/cmd/tlsdiag.go | 0.0% |
| github.com/libp2p/go-libp2p/p2p/security/tls/cmd/tlsdiag/client.go | 0.0% |
| github.com/libp2p/go-libp2p/p2p/security/tls/cmd/tlsdiag/key.go | 0.0% |
| github.com/libp2p/go-libp2p/p2p/security/tls/cmd/tlsdiag/server.go | 0.0% |
| github.com/libp2p/go-libp2p/p2p/security/tls/conn.go | 100.0% |
| github.com/libp2p/go-libp2p/p2p/security/tls/crypto.go | 84.1% |
| github.com/libp2p/go-libp2p/p2p/security/tls/extension.go | 100.0% |
| github.com/libp2p/go-libp2p/p2p/security/tls/transport.go | 87.8% |
| github.com/libp2p/go-libp2p/p2p/test/resource-manager/echo.go | 59.7% |
| github.com/libp2p/go-libp2p/p2p/transport/internal/quicutils/tracer.go | 59.5% |
| github.com/libp2p/go-libp2p/p2p/transport/quic/cmd/client/main.go | 0.0% |
| github.com/libp2p/go-libp2p/p2p/transport/quic/cmd/server/main.go | 0.0% |
| github.com/libp2p/go-libp2p/p2p/transport/quic/conn.go | 90.0% |
| github.com/libp2p/go-libp2p/p2p/transport/quic/listener.go | 86.2% |
| github.com/libp2p/go-libp2p/p2p/transport/quic/options.go | 60.0% |
| github.com/libp2p/go-libp2p/p2p/transport/quic/quic_multiaddr.go | 77.8% |
| github.com/libp2p/go-libp2p/p2p/transport/quic/reuse.go | 95.8% |
| github.com/libp2p/go-libp2p/p2p/transport/quic/stream.go | 87.5% |
| github.com/libp2p/go-libp2p/p2p/transport/quic/tracer_metrics.go | 20.9% |
| github.com/libp2p/go-libp2p/p2p/transport/quic/transport.go | 73.8% |
| github.com/libp2p/go-libp2p/p2p/transport/tcp/metrics.go | 41.3% |
| github.com/libp2p/go-libp2p/p2p/transport/tcp/metrics_linux.go | 0.0% |
| github.com/libp2p/go-libp2p/p2p/transport/tcp/reuseport.go | 60.0% |
| github.com/libp2p/go-libp2p/p2p/transport/tcp/tcp.go | 82.1% |
| github.com/libp2p/go-libp2p/p2p/transport/testsuite/stream_suite.go | 81.9% |
| github.com/libp2p/go-libp2p/p2p/transport/testsuite/transport_suite.go | 60.2% |
| github.com/libp2p/go-libp2p/p2p/transport/testsuite/utils_suite.go | 85.7% |
| github.com/libp2p/go-libp2p/p2p/transport/websocket/addrs.go | 79.4% |
| github.com/libp2p/go-libp2p/p2p/transport/websocket/conn.go | 91.5% |
| github.com/libp2p/go-libp2p/p2p/transport/websocket/listener.go | 81.1% |
| github.com/libp2p/go-libp2p/p2p/transport/websocket/websocket.go | 85.7% |
| github.com/libp2p/go-libp2p/p2p/transport/webtransport/cert_manager.go | 87.1% |
| github.com/libp2p/go-libp2p/p2p/transport/webtransport/conn.go | 82.4% |
| github.com/libp2p/go-libp2p/p2p/transport/webtransport/crypto.go | 89.7% |
| github.com/libp2p/go-libp2p/p2p/transport/webtransport/listener.go | 84.0% |
| github.com/libp2p/go-libp2p/p2p/transport/webtransport/multiaddr.go | 86.8% |
| github.com/libp2p/go-libp2p/p2p/transport/webtransport/noise_early_data.go | 100.0% |
| github.com/libp2p/go-libp2p/p2p/transport/webtransport/stream.go | 55.6% |
| github.com/libp2p/go-libp2p/p2p/transport/webtransport/transport.go | 88.5% |
go test -v -shuffle=on -coverprofile=module-coverage.txt -coverpkg=./... ./...
go tool cover -html=module-coverage.txt -o=./cover.htmlThe HTML at this point can be viewed in the browser, I extracted the above data from the DOM and formatted it in markdown HTML for your review: cover.html.zip
They run tests as part of CI/CD and also they use the -shuffle=on flag (which maybe we should consider too) to expose potential interdependencies (state leaking or similar issues) between tests that make them fail/succeed depending on other tests that ran before them if that makes sense.
The library per-se is not testing any routing or fancy stuff like that because that's handled in the respective packages (for example github.com/libp2p/go-libp2p-kad-dht).
This highlights the fact that the core package is very modular and offers the low-level primitives without focusing on the algorithms that application developers like us will integrate/develop separately.
Number of commits by file, generated with GitNStats trimmed up to 10 commits per file
Full version: gitstats.txt.zip
Repository: /Users/alex/CODE/OSS/go-libp2p Branch: master
Commits Path 484 go.mod 447 go.sum 381 package.json 284 p2p/host/basic/basic_host.go 270 p2p/net/swarm/swarm.go 256 p2p/protocol/identify/id.go 252 p2p/net/swarm/swarm_dial.go 195 README.md 194 .gx/lastpubver 175 config/config.go 170 p2p/protocol/identify/id_test.go 152 p2p/net/swarm/swarm_test.go 151 p2p/net/swarm/dial_test.go 149 p2p/host/basic/basic_host_test.go 143 options.go 140 p2p/transport/quic/transport.go 138 p2p/net/swarm/swarm_listen.go 123 p2p/net/mock/mock_test.go 117 defaults.go 116 p2p/protocol/identify/obsaddr.go 114 p2p/net/swarm/swarm_conn.go 114 examples/pubsub/chat/go.sum 110 examples/go.sum 107 examples/ipfs-camp-2019/go.sum 105 p2p/net/mock/mock_peernet.go 105 p2p/net/mock/mock_net.go 105 p2p/host/autonat/svc.go 103 p2p/net/connmgr/connmgr.go 101 p2p/host/routed/routed.go 101 p2p/protocol/internal/circuitv1-deprecated/relay.go 100 p2p/net/swarm/swarm_addr_test.go 100 p2p/host/autonat/autonat.go 100 p2p/transport/quic/conn_test.go 99 p2p/net/swarm/limiter.go 99 p2p/transport/tcp/tcp.go 96 p2p/net/mock/mock_conn.go 95 p2p/net/mock/mock_stream.go 95 p2p/protocol/identify/obsaddr_test.go 94 p2p/test/backpressure/backpressure_test.go 94 p2p/net/swarm/simul_test.go 93 p2p/net/swarm/swarm_notif_test.go 93 p2p/host/peerstore/pstoremem/addr_book.go 89 p2p/test/reconnects/reconnect_test.go 89 examples/echo/main.go 88 p2p/net/swarm/testing/testing.go 87 p2p/net/mock/mock_notif_test.go 86 p2p/net/swarm/limiter_test.go 86 p2p/host/basic/natmgr.go 86 examples/ipfs-camp-2019/go.mod 85 p2p/net/swarm/swarm_stream.go 85 libp2p_test.go 84 p2p/transport/websocket/websocket.go 83 p2p/protocol/ping/ping.go 81 p2p/net/swarm/swarm_net_test.go 81 examples/go.mod 80 p2p/net/swarm/peers_test.go 79 libp2p.go 77 p2p/net/swarm/swarm_net.go 76 p2p/transport/quic/listener.go 76 p2p/host/autonat/autonat_test.go 75 p2p/host/autonat/svc_test.go 75 p2p/host/resource-manager/scope.go 74 p2p/net/swarm/swarm_addr.go 74 p2p/net/swarm/dial_sync.go 74 examples/multipro/echo.go 74 p2p/host/peerstore/peerstore.go 72 p2p/protocol/ping/ping_test.go 72 p2p/host/peerstore/pstoreds/addr_book.go 72 p2p/security/noise/handshake.go 71 p2p/net/upgrader/upgrader.go 69 p2p/net/upgrader/listener_test.go 68 examples/multipro/ping.go 68 p2p/host/resource-manager/rcmgr.go 67 p2p/net/mock/mock_link.go 67 package-list.json 66 p2p/net/connmgr/connmgr_test.go 66 examples/multipro/node.go 65 config/security.go 65 p2p/security/noise/transport_test.go 64 p2p/net/mock/interface.go 64 examples/multipro/main.go 63 p2p/discovery/mdns_legacy/mdns.go 63 p2p/host/autorelay/relay_finder.go 63 p2p/host/peerstore/test/addr_book_suite.go 62 .travis.yml 62 examples/pubsub/chat/go.mod 62 p2p/security/noise/transport.go 61 p2p/protocol/identify/pb/identify.pb.go 61 p2p/security/tls/transport_test.go 60 p2p/host/autonat/client.go 59 examples/http-proxy/proxy.go 59 config/muxer.go 58 p2p/net/conn/dial.go 58 config/muxer_test.go 58 p2p/transport/tcp/tcp_test.go 56 p2p/net/swarm/dial_sync_test.go 56 config/reflection_magic.go 56 p2p/protocol/internal/circuitv1-deprecated/relay_test.go 55 p2p/transport/quic/conn.go 55 p2p/net/mock/mock.go 55 p2p/host/blank/blank.go 55 p2p/security/noise/protocol.go 55 p2p/net/upgrader/listener.go 54 p2p/host/peerstore/test/peerstore_suite.go 53 p2p/net/swarm/addr/addr.go 53 examples/echo/README.md 53 version.json 53 p2p/security/tls/crypto.go 52 p2p/discovery/backoff/backoffcache_test.go 52 p2p/muxer/yamux/transport.go 52 config/constructor_types.go 52 examples/chat/chat.go 51 p2p/net/swarm/dial_worker_test.go 50 p2p/host/autonat/notify.go 50 config/transport.go 50 p2p/net/nat/nat.go 50 p2p/security/tls/transport.go 49 p2p/discovery/routing/routing.go 49 examples/README.md 49 p2p/protocol/identify/id_delta.go 49 p2p/protocol/identify/id_glass_test.go 49 p2p/host/eventbus/basic.go 48 examples/libp2p-host/host.go 48 examples/multipro/pb/p2p.proto 48 p2p/transport/webtransport/transport.go 47 p2p/discovery/routing/routing_test.go 47 examples/libp2p-host/README.md 47 config/transport_test.go 47 p2p/net/upgrader/upgrader_test.go 46 p2p/net/conn/listen.go 46 examples/multipro/README.md 46 p2p/transport/quic/listener_test.go 46 p2p/protocol/internal/circuitv1-deprecated/dial.go 45 examples/http-proxy/README.md 45 p2p/protocol/identify/peer_loop.go 45 p2p/host/eventbus/basic_test.go 44 p2p/net/conngater/conngater.go 44 p2p/transport/websocket/websocket_test.go 44 p2p/transport/websocket/listener.go 43 p2p/net/mock/mock_printer.go 43 examples/multipro/pb/p2p.pb.go 43 p2p/host/autonat/proto.go 42 p2p/protocol/identify/pb/identify.proto 42 p2p/host/autonat/options.go 42 p2p/protocol/identify/peer_loop_test.go 42 p2p/discovery/backoff/backoffcache.go 42 examples/chat/README.md 42 p2p/host/peerstore/pstoreds/ds_test.go 41 .gitignore 41 examples/ipfs-camp-2019/06-Pubsub/main.go 41 examples/ipfs-camp-2019/07-Messaging/main.go 41 examples/ipfs-camp-2019/08-End/main.go 41 p2p/discovery/backoff/backoffconnector_test.go 41 p2p/net/connmgr/decay_test.go 41 examples/multipro/AUTHORS 41 examples/multipro/LICENSE 40 examples/relay/main.go 40 p2p/discovery/mdns/mdns_test.go 40 p2p/transport/quic/transport_test.go 40 p2p/host/autorelay/options.go 40 p2p/security/noise/session.go 39 p2p/host/host.go 39 p2p/host/autorelay/autorelay_test.go 39 p2p/protocol/circuitv1/relay/relay.go 39 p2p/host/peerstore/pstoreds/peerstore.go 39 p2p/security/noise/benchmark_test.go 38 p2p/protocol/circuitv2/relay/relay.go 38 p2p/discovery/backoff/backoff.go 38 .github/ISSUE_TEMPLATE/release.md 38 p2p/host/autonat/dialpolicy_test.go 38 .github/workflows/go-check.yml 38 .github/workflows/go-test.yml 38 examples/routed-echo/main.go 38 p2p/host/autonat/test/autonat_test.go 38 p2p/transport/quic/reuse_test.go 38 p2p/security/noise/rw.go 38 core/crypto/key.go 37 p2p/protocol/circuitv2/client/reservation_test.go 37 examples/chat-with-mdns/main.go 37 examples/ipfs-camp-2019/05-Discovery/main.go 37 p2p/host/autonat/interface.go 37 p2p/net/connmgr/bench_test.go 37 examples/pubsub/chat/main.go 37 p2p/transport/websocket/conn.go 37 p2p/transport/quic/reuse_base.go 37 p2p/protocol/internal/circuitv1-deprecated/conn.go 36 p2p/test/util/util.go 36 p2p/host/autonat/pb/autonat.pb.go 36 p2p/discovery/util/util.go 36 p2p/discovery/mocks/mocks.go 36 p2p/net/swarm/swarm_transport.go 36 p2p/net/swarm/addr/addr_test.go 36 p2p/discovery/mdns/mdns.go 36 examples/chat-with-mdns/mdns.go 36 p2p/transport/quic/stream.go 36 p2p/muxer/mplex/conn.go 35 p2p/nat/nat.go 35 p2p/net/interface.go 35 p2p/protocol/relay/relay.go 35 p2p/discovery/backoff/backoffconnector.go 35 p2p/protocol/identify/pb/Makefile 35 p2p/host/pstoremanager/pstoremanager_test.go 35 examples/chat-with-rendezvous/README.md 35 p2p/host/autonat/dialpolicy.go 35 p2p/protocol/identify/opts.go 35 scripts/mkreleaselog 35 examples/chat-with-rendezvous/chat.go 35 examples/chat/chat_test.go 35 examples/ipfs-camp-2019/03-Muxing-Encryption/main.go 35 p2p/net/swarm/transport_test.go 35 p2p/host/peerstore/pstoremem/protobook.go 35 p2p/host/resource-manager/limit.go 35 p2p/protocol/internal/circuitv1-deprecated/transport_test.go 34 p2p/net/conn/dial_test.go 34 p2p/discovery/backoff/backoff_test.go 34 .github/workflows/automerge.yml 34 p2p/net/README.md 34 p2p/net/mock/ratelimiter.go 34 examples/routed-echo/bootstrap.go 34 options_filter.go 34 p2p/protocol/identify/id_push.go 34 examples/chat-with-mdns/README.md 34 p2p/net/connmgr/options.go 34 p2p/net/connmgr/decay.go 34 examples/ipfs-camp-2019/02-Multiaddrs/main.go 34 examples/pubsub/chat/README.md 34 config/reflection_magic_test.go 34 error_util.go 34 p2p/transport/quic/cmd/client/main.go 34 p2p/protocol/holepunch/svc.go 34 p2p/protocol/holepunch/holepunch_test.go 34 limits.go 34 p2p/host/resource-manager/rcmgr_test.go 34 p2p/net/conn-security-multistream/ssms_test.go 34 p2p/transport/webtransport/transport_test.go 33 examples/echo/main_test.go 33 examples/ipfs-camp-2019/07-Messaging/pubsub.go 33 examples/ipfs-camp-2019/08-End/protocol.go 33 examples/ipfs-camp-2019/08-End/pubsub.go 33 p2p/host/autorelay/host.go 33 p2p/host/basic/peer_connectedness_test.go 33 examples/ipfs-camp-2019/01-Transports/main.go 33 p2p/transport/tcp/reuseport.go 33 p2p/transport/tcp/metrics.go 33 p2p/transport/webtransport/listener.go 32 p2p/host/autonat/pb/autonat.proto 32 .github/ISSUE_TEMPLATE/feature.md 32 .github/ISSUE_TEMPLATE/enhancement.md 32 p2p/net/conn/conn.go 32 examples/ipfs-camp-2019/07-Messaging/protocol.go 32 examples/pubsub/chat/ui.go 32 examples/relay/main_test.go 32 p2p/host/relaysvc/relay.go 32 p2p/net/conngater/conngater_test.go 32 p2p/net/mock/complement.go 32 p2p/host/basic/peer_connectedness.go 32 .github/workflows/upstream.yml 32 examples/ipfs-camp-2019/05-Discovery/protocol.go 32 examples/ipfs-camp-2019/06-Pubsub/protocol.go 32 p2p/protocol/circuitv1/relay/options.go 32 p2p/discovery/mdns_legacy/mdns_test.go 32 p2p/transport/quic/tracer_metrics.go 32 p2p/net/swarm/addrs.go 32 p2p/net/pnet/psk_conn.go 32 p2p/security/noise/crypto_test.go 32 p2p/net/upgrader/conn.go 32 p2p/protocol/internal/circuitv1-deprecated/listen.go 32 p2p/protocol/internal/circuitv1-deprecated/util.go 32 core/crypto/key_test.go 32 p2p/protocol/internal/circuitv1-deprecated/transport.go 32 p2p/host/resource-manager/scope_test.go 31 p2p/protocol/holepunch/pb/holepunch.proto 31 p2p/protocol/holepunch/pb/holepunch.pb.go 31 .github/ISSUE_TEMPLATE/question.md 31 .github/ISSUE_TEMPLATE/doc.md 31 .github/ISSUE_TEMPLATE/bug-report.md 31 examples/ipfs-camp-2019/07-Messaging/chat.pb.go 31 examples/ipfs-camp-2019/08-End/chat.pb.go 31 examples/pubsub/chat/chatroom.go 31 examples/routed-echo/README.md 31 examples/testutils/logharness.go 31 p2p/host/autonat/test/dummy.go 31 p2p/host/pstoremanager/mock_peerstore_test.go 31 p2p/host/pstoremanager/pstoremanager.go 31 p2p/host/autorelay/autorelay.go 31 .github/workflows/interop.yml 31 p2p/protocol/holepunch/tracer.go 31 p2p/net/swarm/testing/testing_test.go 31 p2p/transport/testsuite/stream_suite.go 31 p2p/security/noise/crypto.go 31 p2p/security/noise/ik_handshake.go 30 examples/ipfs-camp-2019/02-Multiaddrs/.gitignore 30 examples/ipfs-camp-2019/03-Muxing-Encryption/.gitignore 30 examples/ipfs-camp-2019/05-Discovery/.gitignore 30 examples/ipfs-camp-2019/05-Discovery/README.md 30 p2p/protocol/circuitv1/pb/circuitv1.proto 30 p2p/protocol/circuitv1/pb/circuitv1.pb.go 30 examples/ipfs-camp-2019/07-Messaging/README.md 30 examples/ipfs-camp-2019/07-Messaging/chat.proto 30 examples/ipfs-camp-2019/08-End/.gitignore 30 examples/ipfs-camp-2019/08-End/chat.proto 30 examples/ipfs-camp-2019/README.md 30 examples/libp2p-host/.gitignore 30 examples/libp2p-host/host_test.go 30 examples/multipro/.gitignore 30 examples/multipro/main_test.go 30 examples/pubsub/README.md 30 examples/pubsub/chat/.gitignore 30 examples/pubsub/chat/chat-example.gif 30 examples/relay/.gitignore 30 examples/routed-echo/.gitignore 30 examples/testutils/net.go 30 p2p/protocol/circuitv1/pb/Makefile 30 .github/workflows/release-check.yml 30 .github/workflows/releaser.yml 30 examples/chat-with-mdns/.gitignore 30 examples/chat-with-mdns/flags.go 30 examples/chat-with-rendezvous/.gitignore 30 examples/chat-with-rendezvous/flags.go 30 examples/chat/.gitignore 30 examples/echo/.gitignore 30 examples/http-proxy/.gitignore 30 examples/ipfs-camp-2019/01-Transports/.gitignore 30 examples/ipfs-camp-2019/06-Pubsub/.gitignore 30 examples/ipfs-camp-2019/06-Pubsub/README.md 30 examples/ipfs-camp-2019/07-Messaging/.gitignore 30 p2p/host/autonat/pb/Makefile 30 config/config_test.go 30 p2p/transport/quic/cmd/server/main.go 30 p2p/protocol/holepunch/holepuncher.go 30 p2p/host/peerstore/pstoreds/addr_book_gc.go 30 p2p/muxer/yamux/stream.go 29 p2p/net/conn/interface.go 29 p2p/net/swarm/dial_error.go 29 .codecov.yml 29 p2p/protocol/holepunch/pb/Makefile 29 p2p/test/backpressure/backpressure.go 29 p2p/test/reconnects/reconnect.go 29 p2p/protocol/identify/obsaddr_glass_test.go 29 p2p/transport/websocket/addrs.go 29 p2p/host/blank/peer_connectedness_test.go 29 p2p/host/peerstore/pstoreds/protobook.go 29 p2p/host/peerstore/test/keybook_suite.go 29 p2p/muxer/yamux/conn.go 29 p2p/net/conn-security-multistream/ssms.go 29 p2p/host/resource-manager/trace.go 29 p2p/test/resource-manager/rcmgr_test.go 28 p2p/protocol/relay/relay_test.go 28 LICENSE 28 Makefile 28 p2p/protocol/circuitv2/client/transport.go 28 .github/workflows/tagpush.yml 28 p2p/protocol/circuitv2/client/dial.go 28 p2p/net/swarm/dial_worker.go 28 p2p/host/blank/peer_connectedness.go 28 p2p/net/pnet/protector.go 28 p2p/host/peerstore/pstoremem/metadata.go 28 p2p/host/resource-manager/limit_defaults.go 28 p2p/transport/websocket/conn_browser.go 27 p2p/protocol/circuitv2/client/client.go 27 p2p/transport/tcp/metrics_general.go 27 p2p/net/swarm/util_test.go 27 p2p/net/connmgr/watchdog_no_cgo.go 27 p2p/net/connmgr/watchdog_cgo.go 27 p2p/host/peerstore/pstoremem/inmem_test.go 27 p2p/host/peerstore/pstoreds/addr_book_gc_test.go 27 p2p/muxer/yamux/transport_test.go 27 p2p/security/noise/integration_test.go 27 p2p/muxer/muxer-multistream/multistream.go 27 p2p/host/peerstore/metrics_test.go 26 p2p/test/util/key.go 26 p2p/crypto/secio/protocol.go 26 p2p/protocol/circuitv2/client/reservation.go 26 p2p/protocol/circuitv2/client/handlers.go 26 p2p/transport/websocket/addrs_test.go 26 p2p/protocol/identify/id_go118.go 26 p2p/protocol/holepunch/util.go 26 p2p/host/peerstore/pstoreds/keybook.go 26 p2p/muxer/mplex/stream.go 26 core/sec/insecure/insecure.go 25 p2p/protocol/circuitv2/util/pbconv.go 25 p2p/net/conn/secure_conn.go 25 p2p/transport/quic/mock_connection_gater_test.go 25 p2p/transport/tcp/metrics_linux.go 25 p2p/transport/tcp/metrics_darwin.go 25 p2p/protocol/identify/id_go117.go 25 p2p/net/pnet/psk_conn_test.go 25 p2p/net/reuseport/transport_test.go 25 p2p/host/eventbus/opts.go 25 p2p/muxer/mplex/transport.go 24 p2p/peer/peer.go 24 p2p/protocol/circuitv2/relay/relay_test.go 24 p2p/transport/tcp/metrics_windows.go 24 p2p/host/resource-manager/obs/stats.go 24 p2p/net/upgrader/threshold.go 24 p2p/host/peerstore/interface.go 24 core/crypto/ecdsa.go 24 p2p/host/peerstore/test/benchmarks_suite.go 24 p2p/test/resource-manager/echo.go 23 p2p/net/conn/secure_conn_test.go 23 p2p/host/peerstore/queue/queue_test.go 23 p2p/protocol/circuitv2/client/conn.go 23 p2p/transport/quic/quic_multiaddr.go 23 .github/actions/go-test-setup/action.yml 23 p2p/host/peerstore/pstoreds/metadata.go 23 core/peer/peer.go 23 core/crypto/ed25519.go 23 p2p/host/resource-manager/README.md 23 p2p/test/resource-manager/echo_test.go 22 p2p/protocol/mux.go 22 p2p/transport/quic/quic_multiaddr_test.go 22 p2p/host/peerstore/peerinfo.go 22 p2p/host/peerstore/metrics.go 22 p2p/security/noise/pb/payload.pb.go 22 p2p/security/noise/pb/payload.proto 22 p2p/muxer/mplex/transport_test.go 21 p2p/protocol/circuitv2/proto/voucher.go 21 p2p/protocol/circuitv2/proto/voucher_test.go 21 p2p/protocol/circuitv2/relay/acl.go 21 p2p/protocol/circuitv2/client/listen.go 21 p2p/protocol/circuitv2/relay/constraints_test.go 21 p2p/protocol/circuitv2/relay/constraints.go 21 p2p/protocol/holepunch/coordination.go 21 p2p/transport/websocket/LICENSE-MIT 21 p2p/transport/websocket/LICENSE-APACHE 21 p2p/net/mock/log2.txt 21 p2p/net/reuseport/reuseport.go 21 p2p/transport/testsuite/transport_suite.go 21 p2p/security/tls/cmd/tlsdiag/key.go 21 p2p/net/upgrader/gater_test.go 21 core/network/network.go 20 p2p/net/filter/filter.go 20 p2p/crypto/secio/rw.go 20 p2p/protocol/circuitv2/relay/resources.go 20 p2p/protocol/circuitv2/util/io.go 20 p2p/protocol/circuitv2/pb/circuit.proto 20 p2p/protocol/circuitv2/pb/circuit.pb.go 20 p2p/host/peerstore/queue/sync.go 20 p2p/host/peerstore/pstoremem/keybook.go 20 p2p/host/resource-manager/obs/grafana-dashboards/resource-manager.json 20 p2p/security/tls/extension_test.go 20 p2p/security/tls/conn.go 19 p2p/net/conn/conn_test.go 19 p2p/protocol/circuitv2/relay/options.go 19 p2p/protocol/circuitv2/pb/Makefile 19 p2p/protocol/circuitv2/pb/voucher.pb.go 19 p2p/peer/peerstore.go 19 p2p/net/transport/tcp.go 19 p2p/crypto/key.go 19 p2p/protocol/circuitv2/pb/voucher.proto 19 p2p/protocol/circuitv2/proto/protocol.go 19 p2p/net/nat/mapping.go 19 p2p/host/peerstore/pstoremem/peerstore.go 19 p2p/security/noise/session_test.go 19 p2p/security/tls/cmd/tlsdiag/client.go 19 p2p/security/tls/cmd/tlsdiag.go 19 core/crypto/key_openssl.go 18 p2p/host/autorelay/addrsplosion_test.go 18 p2p/protocol/circuitv2/relay/compat_test.go 18 p2p/host/peerstore/queue/distance.go 18 p2p/host/resource-manager/allowlist.go 18 p2p/security/tls/extension.go 18 p2p/security/tls/cmd/tlsdiag/server.go 18 core/sec/insecure/insecure_test.go 18 core/crypto/key_not_openssl.go 17 p2p/net/transport/transport.go 17 p2p/host/peerstore/pb/pstore.pb.go 17 p2p/host/resource-manager/allowlist_test.go 17 p2p/security/noise/LICENSE.md 17 p2p/security/noise/pb/Makefile 17 core/peer/peer_test.go 17 p2p/host/peerstore/peerinfo_test.go 17 p2p/host/peerstore/pb/custom.go 17 core/peerstore/peerstore.go 17 core/peer/record.go 17 p2p/security/noise/session_transport.go 17 p2p/transport/internal/quicutils/tracer.go 16 p2p/metrics/stream/metered_test.go 16 p2p/host/autorelay/addrsplosion.go 16 testutil/gen.go 16 p2p/crypto/rsa.go 16 p2p/host/peerstore/addr/addrsrcs_test.go 16 p2p/host/peerstore/addr/addrsrcs.go 16 p2p/host/autorelay/relay.go 16 p2p/transport/quic/libp2pquic_suite_test.go 16 p2p/net/reuseport/reuseport_test.go 16 p2p/security/noise/xx/XX_test.go 16 p2p/security/noise/xx/XX.noise.go 16 p2p/security/noise/ik/IK.noise.go 16 p2p/net/reuseport/dial.go 16 core/crypto/rsa_go.go 16 core/sec/insecure/pb/plaintext.pb.go 16 core/peer/addrinfo.go 16 p2p/host/resource-manager/metrics.go 16 p2p/protocol/internal/circuitv1-deprecated/pb/relay.pb.go 16 p2p/transport/websocket/browser_integration_native_test.go 15 p2p/host/peerstore/peer.go 15 testutil/identity.go 15 examples/justtcp/main.go 15 p2p/peer/queue/sync.go 15 p2p/peer/peer_test.go 15 p2p/crypto/pb/crypto.pb.go 15 p2p/host/relay/autorelay.go 15 p2p/net/reuseport/reuseport_posix.go 15 p2p/net/reuseport/listen.go 15 p2p/net/reuseport/transport.go 15 p2p/security/tls/LICENSE.md 15 core/crypto/rsa_openssl.go 15 p2p/host/resource-manager/limit_config_test.go 15 p2p/transport/webtransport/conn.go 15 p2p/discovery/mdns.go 14 p2p/metrics/bw_stats.go 14 p2p/peer/queue/queue_test.go 14 p2p/peer/addr_manager.go 14 p2p/crypto/secio/pb/spipe.pb.go 14 p2p/net/reuseport/dialer.go 14 examples/multipro/pb/readme.md 14 p2p/transport/testsuite/utils_suite.go 14 core/crypto/rsa_test.go 14 p2p/host/resource-manager/limit_dynamic.go 14 p2p/host/resource-manager/extapi.go 14 core/transport/transport.go 14 p2p/transport/webtransport/cert_manager.go 14 p2p/protocol/internal/circuitv1-deprecated/notify.go 13 p2p/peer/addr_manager_test.go 13 p2p/crypto/secio/interface.go 13 p2p/crypto/secio/al.go 13 p2p/host/peerstore/peer_test.go 13 p2p/host/peerstore/pb/pstorepb_test.go 13 p2p/host/resource-manager/limit_test.go 13 p2p/net/reuseport/reuseport_plan9.go 13 core/metrics/bandwidth_test.go 13 core/event/addrs.go 13 p2p/host/peerstore/pstoreds/cyclic_batch.go 12 p2p/metrics/stream/metered.go 12 p2p/protocol/mux_test.go 12 p2p/host/match.go 12 p2p/peer/addr/addrsrcs_test.go 12 p2p/peer/addr/addrsrcs.go 12 p2p/host/peerstore/queue/interface.go 12 p2p/transport/quic/crypto.go 12 p2p/transport/internal/quicutils/tracer_test.go 12 p2p/host/peerstore/pb/pstore.proto 12 p2p/host/resource-manager/obs/grafana-dashboards/README.md 12 core/crypto/openssl_common.go 12 core/crypto/secp256k1.go 12 core/metrics/bandwidth.go 12 p2p/host/resource-manager/limit_config.go 12 p2p/host/peerstore/pstoreds/cache.go 12 p2p/host/peerstore/pb/Makefile 12 core/event/network.go 12 p2p/host/relay/autorelay_test.go 12 p2p/transport/websocket/browser_integration_browser_test.go 11 p2p/metrics/interface.go 11 p2p/metrics/conn/conn.go 11 p2p/peer/queue/distance.go 11 loggables/loggables.go 11 examples/protocol-multiplexing-with-multicodecs/README.md 11 examples/protocol-multiplexing-with-multicodecs/main.go 11 core/sec/insecure/pb/plaintext.proto 11 p2p/net/reuseport/singledialer.go 11 core/routing/query.go 11 p2p/host/resource-manager/limit_static.go 11 p2p/protocol/internal/circuitv1-deprecated/pb/relay.proto 11 p2p/protocol/internal/circuitv1-deprecated/pb/Makefile 11 core/network/conn.go 10 p2p/net/swarm/addr/filter.go 10 p2p/net/transport/utp.go 10 p2p/nat/mapping.go 10 core/sec/insecure/pb/Makefile 10 examples/multipro/protocol.go 10 p2p/host/resource-manager/obs/stats_test.go 10 p2p/host/resource-manager/limit_config_test.json 10 p2p/host/resource-manager/error.go 10 core/network/context.go
2. Components: Make a list of LibP2P components we plan to reference / use / adopt in the foreseeable future (e.g. yes to transport layer, no to kad-dht).
This question requires a deep understanding of the library which unfortunately I don't have -yet-, so take these with a grain of salt.
| Icon | Meaning |
|---|---|
| ✅ | I think we'll use it |
| 🤔 | Not sure, possibly |
| ♊ | We might already have our own implementations, needs some thinkering |
| ❌ | Definitely not |
├── config ✅ ├── core │ ├── canonicallog 🤔(we'll have to probably plug logging into ours) │ ├── connmgr ✅ │ ├── control ❌ │ ├── crypto ♊ │ │ ├── pb │ │ └── test_data │ ├── discovery 🤔 (discovering services offered by peers, depends on our impl.) │ ├── event ✅ │ ├── host ✅ │ ├── internal │ │ └── catch 🤔 (recovering from panics) │ ├── introspection 🤔 (used mostly internally for mocking) │ │ └── pb │ ├── metrics 🤔 (we'll have possibly to plug it in our metrics) │ ├── network ✅ │ │ └── mocks │ ├── peer ✅ │ │ └── pb │ ├── peerstore ✅ (addrbook) │ ├── pnet 🤔 (private networks... perhaps, I need to look deeper) │ ├── protocol ✅ │ ├── record ✅ (messaging) │ │ └── pb │ ├── routing ✅ 💯 │ ├── sec ✅ (security) │ │ └── insecure │ │ └── pb │ ├── test 🤔(depends on how we implement our tests) │ └── transport ✅💯 ├── examples ❌ (only as reference maybe) │ ├── chat ❌ (only as reference maybe) │ ├── chat-with-mdns ❌ (only as reference maybe) │ ├── chat-with-rendezvous ❌ (only as reference maybe) │ ├── echo ❌ (only as reference maybe) │ ├── http-proxy ❌ (only as reference maybe) │ ├── ipfs-camp-2019 ❌ (only as reference maybe) │ │ ├── 01-Transports ❌ (only as reference maybe) │ │ ├── 02-Multiaddrs ❌ (only as reference maybe) │ │ ├── 03-Muxing-Encryption ❌ (only as reference maybe) │ │ ├── 05-Discovery ❌ (only as reference maybe) │ │ ├── 06-Pubsub ❌ (only as reference maybe) │ │ ├── 07-Messaging ❌ (only as reference maybe) │ │ └── 08-End ❌ (only as reference maybe) │ ├── libp2p-host ❌ (only as reference maybe) │ ├── multipro ❌ (only as reference maybe) │ │ └── pb ❌ (only as reference maybe) │ ├── pubsub ❌ (only as reference maybe) │ │ ├── basic-chat-with-rendezvous ❌ (only as reference maybe) │ │ └── chat ❌ (only as reference maybe) │ ├── relay ❌ (only as reference maybe) │ ├── routed-echo ❌ (only as reference maybe) │ └── testutils❌ (only as reference maybe) ├── p2p │ ├── discovery │ │ ├── backoff ✅ │ │ ├── mdns 🤔 (multicast DNS) │ │ ├── mocks 🤔 (depending on our testing strategy) │ │ ├── routing ✅ │ │ └── util 🤔(if we embrace the concept of a peer providing a service, from the Raintree spec it sounds like we might) │ ├── host │ │ ├── autonat 🤔(nat autodiscovery) │ │ │ ├── pb │ │ │ └── test │ │ ├── autorelay 🤔 │ │ ├── basic ✅ (potentally in our initial implementations but we'd soon switch to routed) │ │ ├── blank 🤔 (maybe in testing to simulate particular scenarios) │ │ ├── eventbus ♊ (It looks like a more complex and feature-rich version of our event bus) │ │ ├── peerstore 🤔(depends on our implementation, this could be used as reference or used verbatim, not sure yet) │ │ │ ├── pb │ │ │ ├── pstoreds 🤔 (addrbook: this is functionally equivalent to the below, needs more 🧠) │ │ │ ├── pstoremem 🤔 (addrbook) │ │ │ └── test 🤔 │ │ ├── pstoremanager 🤔 │ │ ├── relaysvc 🤔 (it's used in the basicHost, not sure) │ │ ├── resource-manager ✅💯 (low level interface with the OS) │ │ │ ├── docs │ │ │ └── obs │ │ │ └── grafana-dashboards │ │ └── routed ✅ │ ├── muxer │ │ ├── mplex ✅ │ │ ├── muxer-multistream ✅ │ │ ├── testsuite 🤔 │ │ └── yamux 🤔 │ ├── net ✅ │ │ ├── conn-security-multistream 🤔 (secure multiplexing) │ │ ├── conngater 🤔 (access control) │ │ ├── connmgr ✅ │ │ ├── mock 🤔 (depends on our testing strategy but I think so) │ │ ├── nat ✅ │ │ ├── pnet 🤔 (secure connection upgrader) │ │ ├── reuseport ✅ │ │ ├── swarm ✅💯 (connection muxer) │ │ │ └── testing │ │ └── upgrader ✅ (upgrades conns to secure and mplexed conns) │ ├── protocol │ │ ├── circuitv1 ❌ │ │ │ ├── pb │ │ │ └── relay │ │ ├── circuitv2 ✅ │ │ │ ├── client │ │ │ ├── pb │ │ │ ├── proto │ │ │ ├── relay │ │ │ └── util │ │ ├── holepunch 🤔 (nice feature to have perhaps) │ │ │ └── pb │ │ ├── identify ✅ (provides peer's basic "hello") │ │ │ └── pb │ │ ├── internal ❌ │ │ │ └── circuitv1-deprecated │ │ │ └── pb │ │ └── ping 🤔 (maybe opt-in as in basicHost) │ ├── security │ │ ├── noise 🤔 (it looks like it's used by default in conjunction with tls) │ │ │ └── pb │ │ └── tls ✅💯 │ │ └── cmd │ │ └── tlsdiag │ ├── test ✅ │ │ ├── backpressure │ │ ├── reconnects │ │ ├── resource-manager │ │ └── webtransport │ └── transport │ ├── internal │ │ └── quicutils │ ├── quic ✅💯 │ │ └── cmd │ │ ├── client │ │ └── server │ ├── tcp ✅💯 │ ├── testsuite │ ├── websocket 🤔(enabled by default like quic and tcp, need more 🧠) │ └── webtransport 🤔(if we want to support websockets, probably we should support this too) └── scripts
3. Interfaces & Primitives: Identify & references a list of primitives and interfaces that we plan to use in the M1-M4 milestones. Making mirrors of these in the v1 repo can be done in https://github.com/pokt-network/pocket/issues/347.
It's hard to look at the roadmap and think what we need by milestone.
I'd go as far as M1 for now also because we still haven't decided if we are going to use LibP2P.
We know is that by the end of 2023 Q1 we need to build the word Network in Pocket Network basically.
Sounds easy! (Last famous words)
I'll try to decompose the features/tasks starting from the bottom up:
These should give us a "Basic LocalNet" using LibP2P and also prepare the ground for what comes next
In parallel (if there's capacity) I would also do:
"The network should be able to prioritize particular peers for some specific roles" and "The substrate should allow for higher-order structures to be established on top of it, such that a specific set of peers is organized by role or rank or priority." and "The network should support segmented communication. (This is an outcome of the previous requirement where a set of peers might be interested in communicating only among themselves.)" which are all related using LibP2P primitives
My intuition regarding the latter is that we could leverage the concept of service/content discovery that is widely used in the context of P2P file sharing within the library. It's a hunch I have but it requires some extra thought.
Olshansk
commented
1 year ago I apologize for the latency on the round trip time it took me to ACK and RESPOND to this message. This is an amazing analysis and exactly what I was hoping to see!
covered Gemini using it as an example to swap the routing algorithm in LibP2P but then we moved on from that.
Sorry about that. As discussed offline, let’s hold off the Gemini research right now.
I rushed into summarizing a short version of my findings and that contributed to the fact that my exposition was particularly messy :)
Noted. I very much appreciate you sharing WIP in public! Will aim to do more of that myself as well. 📝
Methodology
Amazing! I would have never guessed go has native tools to get git stats like this. 🧑🎓
-shuffle=on flag (which maybe we should consider too
I added a comment with the IMPROVE tag at the top of my current PR. ✅
The library per-se is not testing any routing or fancy stuff like that because that's handled in the respective packages
Make sure to check out https://www.youtube.com/watch?v=M5gy_-nzcR8. Fancy way to test non fancy code :)
This highlights the fact that the core package is very modular and offers the low-level primitives without focusing on the algorithms that application developers like us will integrate/develop separately.
Amazing ⭐
We know is that by the end of 2023 Q1 we need to build the word Network in Pocket Network basically. Sounds easy! (Last famous words) Sounds like famous first words to me :)
This question requires a deep understanding of the library which unfortunately I don't have -yet-, so take these with a grain of salt.
This is a great way to summarize it and I like the use of emojis to make it easier to read. 😃
Implement the Host interface in our nodes
This 💯seems like the way to go. Offline we discussed about copy-pasting the interface, but now that I have read through all of this, I’m thinking of simply embedding it.
See how I imported & embedded smt.MapStore.
You can create type PocketHost interface that embeds Host (adding the necessary documentation) which lets us use their interface but doesn’t limit us from adding our own additions.
this should work in LocalNet so we need a way to spin up nodes at will (K8O?). Seed nodes should be configurable as well.
Yea, we’ll definitely use k8s operators for it along with tilt.dev.
@okdas is working on it in #186.
Implement Peer discovery & Churn (with tests possibly...)
Tests are super useful, but I personally would focus more on debugging and visibility. Try to imagine a situation where we need to figure how who is seeing whom and see what’s happening. For example, imagine a way of exporting the address book into neo4j at different heights and/or timestamps and then visualizing it?
Depending on the results of this research, wrap our Raintree implementation using LibP2P primitives and/or developing new ones in a way that we can plug the implementation into LibP2P for example is the simplest one My intuition regarding the latter is that we could leverage the concept of service/content discovery that is widely used in the context of P2P file sharing within the library. It's a hunch I have but it requires some extra thought.
Without too much thought, I really like the direction of this idea. Will think / mull on it for a bit and follow up next week. 🤔
Olshansk
commented
1 year ago @deblasis Do you think there's anything else for us to do here?
I feel like you've provided sufficient context for us to start using their interface as a foundation, and we'll learn more along the way.
I think we can close this out unless there was more work you wanted to put here specifically.
deblasis
commented
1 year ago @Olshansk: I have put together a two pager summarizing my findings. Adding it here for the records.
It's super simple, just trying to capture some context and the next steps. If it needs more work LMK.
Other than that I believe that this issue can be closed.
Link: https://docs.google.com/document/d/1cAWdu8tfeVdMc0xBmUzWUQ53hVTgj_Otz7mdURaTZu4/edit?usp=sharing
Olshansk
commented
1 year ago @deblasis A month late, but I finally read the document and really appreciate the summary. In particular, the actionable next steps.
I referenced it in #438, which is an umbrella ticket capturing the follow-up work on the P2P module.
Going to close this out as complete.
Objective
Research LibP2P (in parallel with #16) to determine if and/or how we should use this library in V1 P2P with a focus on:
Origin Document
There has been prior research about LibP2P that determined this was not the best solution for peer discovery and churn as part of the P2P research work: V1 P2P discovery
However, as we continue to tackle RainTree tech debt and prepare to integrate the Persistence module with P2P, we should revisit this research to confirm that LibP2P2's Kademlia algorithm is not the right solution for Peer Discovery and Churn with a focus on:
Additionally, even if we do not leverage LibP2P for Peer Discovery and Churn, we should revisit this library prior to finishing the complete P2P module in case there are other elements of the library we wish to integrate, and if LibP2P will be compatible with the solution that we do land on for Peer Discovery and Churn with a focus on:
Goals
Deliverable
Non-goals / Non-deliverables
Creator: @jessicadaugherty Editor: @Olshansk