search

nspcc-dev / neo-bench

Neo Blockchain Benchmark toolkit
GNU General Public License v3.0
10 stars 3 forks source link

Allow to bench custom network topologies #92

Open fyrchik opened 3 years ago

fyrchik commented 3 years ago

In real networks topology is far from full mesh. It would be nice if we could bench such topologies too. The easiest way to achieve this is to restrict MaxPeers to 2, but we also need to be sure that each node has it's way to RPC node. This can be achieved either with some custom docker network configuration or by explicitly blacklisting IPs inside the container.

roman-khimov commented 3 years ago

I don't think it makes much sense for a network with 4 CNs, but I'd really like to see 7 CNs plus 2 RPC nodes setup with a limit of up to 3-4 connections per node, that'd be interesting. Also, we probably want to have some well-known set of topologies to compare different results against each other.

fyrchik commented 3 years ago

@roman-khimov Actually I already have some interesting results even for 4 CNs with line/star topologies, where one of the nodes tries to initiate change view and/or receives new blocks before the commit messages for them ("invalid signature" error).

The problem with just restricting MaxPeers is less control over the actual topology and thus less deterministic results (e.g. for 7 nodes we can have a single bridge between 2 almost full-mesh clusters or a more balanced variant. Also, the load depends on which node will be connected with the RPC, we can even fail to run bench if the first one is not).

I think the best solution is:

  1. Generate subnet for each connected set of nodes (e.g. to connect 4 nodes in line we have neo_go_12, neo_go_23, neo_go_34 networks with each node except the last belonging to 2 networks + the first one uses default to connect with RPC node).
  2. Have single docker-compose template (this also helps to unify go/mixed/sharp as well as 7+2 setups).
  3. The actual files can be written by hand (or with some generation utility) or generated on the fly based on simple descriptions (star.4, mesh.7 etc.).

(1) also allows to perform traffic shaping on a per-subnet basis (subnets should correspond to different interfaces in containers, though I haven't checked this). We can do (2) separately in #95 .

roman-khimov commented 3 years ago

We can have a hell lot of different connection patterns even for seven nodes. Some of them are unrealistic (like "line" mentioned above), maybe it's interesting to see how things go in this case, but at least initially I'd be more interested in something resembling real network. That's why I'm trying to create some scheme that doesn't need a lot of configuration tweaks, because subnets will require you to provide different seeds for every machine and be careful with connection limits as the node will try to reach those other nodes it'll receive via getaddr. Subnets make the scheme more fragile, I'd say. At the same time, reproducibility is important, so the more control we have the better. Maybe we need to try different approaches and see how they work, but let's start with simpler ones.