onflow / flow-go

A fast, secure, and developer-friendly blockchain built to support the next generation of games, apps, and the digital assets that power them.
GNU Affero General Public License v3.0
532 stars 175 forks source link

[Networking] Explore Alternative Serialization Protocols for LibP2P Multiplexer #4281

Closed franklywatson closed 1 week ago

franklywatson commented 1 year ago

Context

Historical benchmarking results from when the libp2p node was using JSON serialization indicated significant CPU activity related to the QUIC protocol implementation. Since then, the serialization protocol has been switched to CBOR (Concise Binary Object Representation), likely improving performance. However, we would like to explore other options to ensure that we are using the most efficient serialization protocol possible.

Page up
Samples: 2M of event 'cpu-clock', 4000 Hz, Event count (approx.): 77394383961                                                                     
  Children      Self  Shared Object             Symbol                                                                                            

     5.30%     0.00%  app                       [.] github.com/francoispqt/gojay.(*Encoder).ObjectNullEmpty
     5.20%     0.43%  [kernel]                  [k] __softirqentry_text_start
+    5.15%     0.00%  app                       [.] github.com/libp2p/go-yamux/v2.(*Session).handlePing
+    5.05%     0.00%  app                       [.] github.com/libp2p/go-yamux/v2.(*Stream).sendWindowUpdate
     4.86%     0.00%  app                       [.] github.com/lucas-clemente/quic-go/qlog.frame.MarshalJSONObject
+    4.85%     0.00%  app                       [.] github.com/lucas-clemente/quic-go/qlog.(*eventLossTimerSet).Category

Problem Definition

Although the switch to CBOR has likely improved performance compared to JSON, it is essential to continue exploring alternative serialization protocols to ensure that the libp2p node is operating at its optimal efficiency.

Proposed Solution

  1. Research and identify potential alternative serialization protocols to compare against CBOR and JSON.
  2. Implement and benchmark the chosen alternative(s) within the libp2p multiplexer to evaluate their performance.

Benefits

Exploring alternative serialization protocols for the libp2p multiplexer will yield the following benefits:

  1. Potential further reduction in CPU usage and improved performance of the libp2p node.
  2. Increased ability to handle high levels of network traffic.
  3. Enhanced overall efficiency and reliability of the libp2p network.

Definition of Done

  1. Alternative serialization protocols have been researched and identified for comparison against CBOR and JSON.
  2. The chosen alternative(s) have been implemented within our codebase.
  3. Benchmarking results demonstrate the performance of the alternative(s) compared to CBOR and JSON.
  4. The code should be documented appropriately and conform to the Flow coding standards.
  5. All existing tests in flow-go codebase should pass despite of the changes.
  6. Exploring alternatives should not break any existing functionality or use cases of the libp2p node.
yhassanzadeh13 commented 11 months ago

Performed the first round of review on the conducted research and provided comments.

Guitarheroua commented 11 months ago

Duplicate of investigation for the issue on Google Doc

github-actions[bot] commented 2 weeks ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.