Analyze Block Part Distribution in Knuu for Block Proposer and Immediate Connections

[staheri14]

Problem

Based on the traced data obtained from Testground experiments, it has been observed that some of the immediate connections of a block proposer do not receive block parts of the proposal first from the proposer. Instead, they obtain it from another peer. This occurs despite those direct connections of the proposer not having fully utilized their received rate. Hence, in theory, they are expected to receive all the block parts from the proposer, not from an intermediary.

There is a hypothesis that this issue might be unique to Testground. To verify this, we need to analyze the same scenario in Knuu. This issue aims to capture all aspects related to this analysis:

The analysis is based on the consensus_block_parts table.

• It focuses on the block proposer and its immediate connections (for all heights).
• The time at which the immediate connections of the proposer first observe a block part of the proposal is captured, along with the peer who sent it.
• The time at which the block part was sent by the proposer to each of its connections is also captured.

We need to determine:

• If any peer received a block part first from another peer rather than the proposer,
• How long it took for that part to arrive, and
• What the received rate of the peer was at the time of arrival of that part (This part needs some clarification @evan-forbes).

[staheri14]

@evan-forbes Can you please share the plots demonstrating this behaviour in testground.

[evan-forbes]

Here's a plot for a single height where the proposer is quickly queueing all of the block parts, but they are unable to be sent despite not all bandwidth being used.

to reproduce, first run the script that matches all sends and receives, then combine that data with the bandwidth data. How this exact process is done is a bit arbitrary, but the matching can be found https://github.com/celestiaorg/big-blocks-research/blob/evan/trace-3/match_parts.py. I believe I sent some other scripts to you as well that create this plot, but let me know if its still needed