hyperledger / firefly

Hyperledger FireFly is the first open source Supernode: a complete stack for enterprises to build and scale secure Web3 applications. The FireFly API for digital assets, data flows, and blockchain transactions makes it radically faster to build production-ready apps on popular chains and protocols.
https://hyperledger.github.io/firefly
Apache License 2.0
507 stars 208 forks source link

v1.3.1 Performance Testing #1540

Closed EnriqueL8 closed 3 months ago

EnriqueL8 commented 4 months ago

Run the performance tests using the perf-cli, similar to https://github.com/hyperledger/firefly/issues/1470#issuecomment-2077567912

EnriqueL8 commented 4 months ago

Run against 1.3.1 RC 1

Started: 19/07/24 Duration: ~32 hours Git commit: https://github.com/hyperledger/firefly/commit/3a2d40cca759c761ff0bba839ac582da163453d2

Node Configuration 2 FireFly nodes on one virtual server (EC2 m4.xlarge) Entire FireFly stack is local to the server (ie both blockchains, Postgres databases, etc) Single geth node with 2 instances of ethconnect Maximum time to confirm before considering failure = 1 minute

stackJSONPath: /home/ubuntu/.firefly/stacks/newstack/stack.json

wsConfig:
  wsPath: /ws
  readBufferSize: 16000
  writeBufferSize: 16000
  initialDelay: 250ms
  maximumDelay: 30s
  initialConnectAttempts: 5
  heartbeatInterval: 5s

instances:
  - name: long-run
    tests: [{"name": "msg_broadcast", "workers":50},{"name": "msg_private", "workers":50},{"name": "blob_broadcast", "workers":30},{"name": "blob_private", "workers":30},{"name": "custom_ethereum_contract", "workers":20},{"name": "token_mint", "workers":10}]
    length: 500h
    sender: 0
    recipient: 1
    messageOptions:
      longMessage: false
    tokenOptions:
      tokenType: fungible
    contractOptions: {"address": "0x2715cc29d158e2d5261cb33a2eec0c6789e7eb6d"}

Shutdown summary

INFO[2024-07-20T18:09:05.280] Shutdown summary:
INFO[2024-07-20T18:09:05.282]  - Prometheus metric sent_mints_total        = 119006.000000
INFO[2024-07-20T18:09:05.282]  - Prometheus metric sent_mint_errors_total  = 0.000000
INFO[2024-07-20T18:09:05.282]  - Prometheus metric mint_token_balance      = 0.000000
INFO[2024-07-20T18:09:05.282]  - Prometheus metric received_events_total   = 7299508.000000
INFO[2024-07-20T18:09:05.282]  - Prometheus metric incomplete_events_total = 0.000000
INFO[2024-07-20T18:09:05.282]  - Prometheus metric delinquent_msgs_total    = 0.000000
INFO[2024-07-20T18:09:05.282]  - Prometheus metric actions_submitted_total = 3530924.000000
INFO[2024-07-20T18:09:05.282]  - Test duration: 31h42m59.398456728s
INFO[2024-07-20T18:09:05.282]  - Measured actions: 14599012
INFO[2024-07-20T18:09:05.282]  - Measured send TPS: 127.927970
INFO[2024-07-20T18:09:05.282]  - Measured throughput: 127.860299
INFO[2024-07-20T18:09:05.282]  - Measured send duration: min: 7.136251ms, max: 2.688845933s, avg: 132ms
INFO[2024-07-20T18:09:05.283]  - Measured event receiving duration: min: 2.007772501s, max: 18.380322296s, avg: 6.138s
INFO[2024-07-20T18:09:05.283]  - Measured total duration: min: 2.007772501s, max: 18.380322296s, avg: 6.138s

Grafana Dashboard

image image image
EnriqueL8 commented 3 months ago

Happy with the above results and performance tests, no degradation compared to v1.3.0 - closing!