Open deepfire opened 4 years ago
cc @dcoutts, @mrBliss, @CodiePP
Benchmarking remains dependent on being able to extract the transaction list from an incoming block, as it first enters the system.
Do I understand correctly that while running the benchmarks, you connect to the node locally (NodeToClient
) and download blocks via its ChainSyncServer
to find out which blocks have been adopted?
What are you benchmarking exactly? Can you describe the exact points between which you want to measure the time?
@mrBliss, not quite -- all nodes are deployed on separate AWS instances, and the setup is as follows:
The key part is that in this particular setting we're measuring the overall system performance, not internal node performance, so it's rather important for us to see blocks/Txs as soon as possible -- on the explorer node.
After a discussion with Thomas, we worked out that https://github.com/input-output-hk/ouroboros-network/blob/685aff740212b45b7fe5eec4b469eea71b209385/ouroboros-consensus/src/Ouroboros/Consensus/NodeNetwork.hs#L323 is probably going to be the optimal trace for benchmarking needs.
cc @CodiePP
In December, the ChainSync implementation used to have a trace that benchmarking heavily relied upon -- the trace was supplying incoming blocks, including their entire contents, that allowed obtaining Tx lists. That trace was deactivated (no longer triggered in
ouroboros-network
code), and a cut-down one is posted to instead, which only traces block headers.The old trace (
TraceChainSyncServerEvent blk blk
) is defined at https://github.com/input-output-hk/ouroboros-network/blob/bb5ef891587789166e8f292a221e6ba63e0c573e/ouroboros-consensus/src/Ouroboros/Consensus/Node/Tracers.hs#L42, which was produced at https://github.com/input-output-hk/ouroboros-network/blob/bb5ef891587789166e8f292a221e6ba63e0c573e/ouroboros-consensus/src/Ouroboros/Consensus/ChainSyncServer.hs#L107.The new trace (
TraceChainSyncServerEvent blk (Header blk)
) now projects the block to its header: https://github.com/input-output-hk/ouroboros-network/blob/master/ouroboros-consensus/src/Ouroboros/Consensus/ChainSyncServer.hs#L112.The change to project the trace appears to have been made in https://github.com/input-output-hk/ouroboros-network/commit/4f912cb4076f4360d6a2c2e461a2a5cd5989e29b
Benchmarking remains dependent on being able to extract the transaction list from an incoming block, as it first enters the system. Note, that one important constraint that the code (currently) has is operating without context (so far) -- which necessitates that all information required to map a block header to the transaction list must be contained in the trace -- see https://github.com/input-output-hk/cardano-node/blob/master/cardano-node/src/Cardano/Tracing/ToObjectOrphans.hs#L784-L797
So this leaves us with two options (lest we're forced to having to re-correlate blocks with individual transactions from separate traces, which sounds like avoidable accidental complexity..):
TraceChainSyncServerEvent blk blk
in the ChainSync implementation -- so tracing code can reference block contents directly.ToObject
instances, that would allow to resolve block headers to blocks proper -- so tracing code can do the work, as-needed.Option #1 introduces potentially undesirable pinning -- after all, we might want to drop blocks from memory at some point -- in the general case.
Option #2 makes tracing to perform the block header-to-block dereferencing work, which sounds fair, in context of the above goal -- after all that work will only be done with block tracing enabled. On the other hand, benchmarking results would be somewhat affected by this extra work done -- which might or might not be insignificant. Oh well, can't win.