search

hyperledger-caliper / caliper

A blockchain benchmark framework to measure performance of multiple blockchain solutions https://wiki.hyperledger.org/display/caliper
https://hyperledger-caliper.github.io/caliper/
Apache License 2.0
654 stars 403 forks source link

Control more aspects of the generated report #320

Open houqinghui opened 5 years ago

houqinghui commented 5 years ago

For example, I want to have more automated control over the name of the report and where it is saved. It would also be nice to have the report formatted as a CSV or something more data-friendly. I think the datas that Caliper produces should be acessibale to user, and User can present the datas in many visualization methods.

aklenik commented 5 years ago

I would add the following remarks:

  1. Report and detailed data should be differentiated. a) A report is a high-level overview of the performance of the system based on the run tests. It should be brief, easy to read and present. b) Detailed data gathered during a test run serves an other purpose, for example identifying bottlenecks of the backend network.
  2. Both should be exported. An ongoing work is to revise the data flow in Caliper, and make it modular to allow for multiple exporters for example (a high level HTML report exporter, a detailed per-transaction CSV exporter, etc).
  3. Some custom information can already be added to the generated report to distinguish between different test runs: 
    "info" : {
"Version": "1.1.0",
"Size": "2 Orgs with 1 Peer",
"Orderer": "Solo",
"Distribution": "Single Host"
}

Place a top-level info block into your network configuration file (see for example, the end of network/fabric-v1.1/2org1peergoleveldb/fabric-go-tls.json). In that block you can add arbitrary key-value pairs, and they will be added to the generated report as-is.

ig0rsky commented 5 years ago

@aklenik is there a way to extract a detailed transaction log of a test run? For example in json or csv form?

aklenik commented 5 years ago

@ig0rsky The Fabric CCP adapter collects everything that can be observed by a client, give it a try. Based on this you can do your own CSV export after each transaction (maybe using a WriteStream for efficiency reasons).