search

hyperledger / caliper

A blockchain benchmark framework to measure performance of multiple blockchain solutions https://wiki.hyperledger.org/display/caliper
https://hyperledger.github.io/caliper/
Apache License 2.0
642 stars 403 forks source link

Calculate the Execution Time in Hyperledger Caliper #172

Open ml16a6a opened 5 years ago

ml16a6a commented 5 years ago

Hi all, How I can calculate the execution time for every round since I got the latency and throughput !?

aklenik commented 5 years ago

@ml16a6a Well, initially the end function of the user callback module received every transaction statistics, so you could calculate anything you want. However, this was removed.

@jiangyaoguo What was the reason for removing this result propagation with this commit? It mostly fixes things with the DRM benchmark, and changes this line in the local-client.js file.

houqinghui commented 5 years ago

@ml16a6a do you already know how to calculate the execution time for every round? you can add the code var t1 = new Date().getTime(); at the beginning , then add the line t2 = new Date().getTime(); console.log(t2-t1) at the end for every round. if so, you can calculate the time.

houqinghui commented 5 years ago

@aklenik is it possible that the project add the functioin to calculate every round execution time as a optional function?

aklenik commented 5 years ago

@houqinghui @ml16a6a Yes, I think it is possible. If the transaction results are annotated with the round ID, then the reporting module could also calculate the round execution time. Plus we shouldn't discard the individual transaction data when aggregating them, because it prohibits detailed data analysis.

conquerorAlex commented 5 years ago

@aklenik @houqinghui As far as I know, the capliper may use fabric-sdk-node to accomplish transaction. Then when calculating execution time, it consists of three parts, the time of capliper invocation, the time of fabric-sdk-node running and the time of fabric running. Is there a way to calculate the time of capliper invocation and fabric-sdk-node running? In another word, how to calculate the performance loss about using caplier?

aklenik commented 5 years ago

@lenoAlex The transaction timings are measured from the point of submitting a TX through the SDK to the point of receiving the event notification about the TX result. So the overhead of Caliper is not included, and every SDK interaction (checking results, inputs, etc.) will also be part of Fabric client applications, so those should be included in the results.