search

hyperledger / caliper

A blockchain benchmark framework to measure performance of multiple blockchain solutions https://wiki.hyperledger.org/display/caliper
https://hyperledger.github.io/caliper/
Apache License 2.0
650 stars 402 forks source link

How latency is calculated in the caliper? #1143

Closed pooja239 closed 1 year ago

pooja239 commented 3 years ago

Is latency calculated only for the successful transactions? Why latency is not calculated for all the transactions?

Context

I am sending 2000 transaction with send rate varying from 5tps-150tps. I am using sample ethereum network. After a certain send rate, many of the transactions are failed. And the system is able to serve certain number of transactions as given by succ in the table below. The latency is expected to increase exponentially but it remains constant.

I wanted to understand, is the latency calculated only for the successful transaction. Why the latency is not considered for all (succ + fail) transactions?

input tps Succ Fail Send Rate (TPS) Max Latency (s) Min Latency (s) Avg Latency (s) Throughput (TPS)
5 2000 0 5 1.18 1.01 1.07 5
10 2000 0 10.1 13.16 1.03 10.08 9.8
15 1151 849 15.2 50.92 1.06 28.65 11
20 770 1230 20.3 51.03 1.04 27.15 13.4
30 603 1397 30.5 51.08 1.06 27.16 17.2
40 541 1459 41.7 50.75 1.8 27.19 20.4
50 515 1485 52.5 50.85 1.79 27.21 22.5
60 504 1496 62.1 50.97 1.05 26.88 24.1
70              
80 478 1522 85.9 51.11 1.85 27.28 27
90              
100 465 1535 105.1 50.63 1.07 26.85 28.6
125 460 1540 134.9 50.71 1.24 26.78 30.8
150 451 1549 172 50.93 1.75 27.14 32.3

Expected Behavior

latency should increase exponentially

Actual Behavior

latency is saturating

Possible Fix

Steps to Reproduce

  1. send rate increases from 1-100tps for sample ethereum network
  2. tx number - 2000

Existing issues

caliper v0.4.2

Context

Your Environment

fatmakoubaa commented 3 years ago

I guess that you got this error because you are using one worker.

fatmakoubaa commented 3 years ago

The latency is calculated for only successful transactions.

stale[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.