hyperledger-caliper / caliper

A blockchain benchmark framework to measure performance of multiple blockchain solutions https://wiki.hyperledger.org/display/caliper
https://hyperledger-caliper.github.io/caliper/
Apache License 2.0
654 stars 405 forks source link

Send rate is not the value I set #1188

Open Bear-hub opened 2 years ago

Bear-hub commented 2 years ago

Context

When I set the fixed-rate to 200 TPS, the actual fixed-rate is more than 600 TPS. This problem occurs when the log reports a timeout error. So I have two questions in total, one is how to determine the maximum TPS of different contracts, the other is when the timeout error will occur, and whether the TPS measured in the background of the timeout error is accurate.

Actual Behavior

myAssetBenchmark.yaml

test:
    name: basic-contract-benchmark
    description: test benchmark
    workers:
      type: local
      number: 2
    rounds:
      - label: updateIot
        description: Update asset benchmark
        txDuration: 30
        rateControl:
          type: fixed-rate
          opts:
            tps: 200
        workload:
          module: workload/updateIot.js
          arguments:
            assets: 5
            contractId: iot
monitors:
  resource:
  - module: docker
    options:
      interval: 5 
      containers:
      - all

Context

I have two questions in total, one is how to determine the maximum TPS of different contracts, the other is when the timeout error will occur, and whether the TPS measured in the background of the timeout error is accurate.

Your Environment

davidkel commented 2 years ago

So you are saying that it achieves the specified tps value if no timeout error occurs ? but if you get a timeout then the reported tps is wrong ? Can you provide more details, such as the report output showing the issue with the associated benchmark file and the actual errors being reported. The rate controller manages the rate based on actual submissions so shouldn't be distinguishing between submissions that subsequently fail vs ones that pass

stale[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

stale[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.