Improve uTP testing

[pipermerriam]

What is wrong

Currently uTP is the most difficult part of our protocol for implementations to get correct. Getting things working in the "normal" case appears to be reasonably. There are however a lot of cases which are harder to get right which show up in live environments such as:

• out of order packets
• dropped packets

How can this be fixed

We need a framework for testing uTP implementations in clients. I believe that we need something that allows us to have some full level of control over what packets are sent and in what order between two running clients. Maybe a JSON-RPC endpoint that accepts a payload with the following:

• Inbound vs Outbound
  • should the client initiate the connection or listen for an inbound connection.
• Packets & Timing
  • A list of packets that should be sent and the timing/delay for when each one should be sent.
• Expected outcomes
  • Should the transfer complete or fail and if so what should the outcome be.

All of this is just intended to be a starting point for an idea of how to test this. The real details will need to be determined by whomever decides to implement this.

[kdeme]

In our nimbus-eth1/fluffy CI we have a uTP testing setup which is using a network simulator to add packet delays, packet drops, lower bandwidth, etc.

It is Portal independent, and just sets up uTP connections over discovery v5 (but in theory could be any transport even). In order to set up these connections and be able to test them we just introduced some basic uTP RPC methods: https://github.com/status-im/nimbus-eth1/blob/master/fluffy/tools/utp_testing/utp_test_rpc_calls.nim#L14-L18

A docker image is build of an uTP application providing these, and these are run with a docker image setup that has this network simulator, see details of setup here: https://github.com/status-im/nimbus-eth1/blob/master/fluffy/tools/utp_testing/README.md

I remember that Trin took over this same setup and does the same or similar testing: https://github.com/ethereum/trin/tree/master/utp-testing, I assume thing is still tested also for Trin.

We might want to link this system somewhere so that other clients can do the same/similar testing, but I agree that a hive like setup using this would even be better to have cross client testing of this.

[ogenev]

I remember that Trin took over this same setup and does the same or similar testing: https://github.com/ethereum/trin/tree/master/utp-testing, I assume thing is still tested also for Trin.

Yes, for Trin we use the same network simulator which allows us to test unordered packets, delays, and drops but it tests only the internal Trin uTP implementation as sender and receiver.