search

ooibc88 / blockbench

BLOCKBENCH: A Framework for Analyzing Private Blockchains. Blockbench contains workloads for measuring the data processing performance, and workloads for understanding the performance of different layers of Blockchain systems.
https://www.comp.nus.edu.sg/~dbsystem/fabricsharp/#/blockbench
Apache License 2.0
391 stars 174 forks source link

How to use the results to get or calculate throughput and latency as the paper #57

Open yolo-612 opened 5 years ago

yolo-612 commented 5 years ago

i run ethereum's macro benchmark kv_store and run./run_bench.sh 2 4 2 10 and i got some results as follow, please help me how to use these data to calculate the throughput and latency as the paper? Results: client_192.168.43.246: https://pastebin.com/uBvX84V3 client_192.168.43.162: https://pastebin.com/USRHRKUy eth_log: https://pastebin.com/pZcUCTvp

how to get ?? thank you

jonghwaC commented 5 years ago

Did you solve it?

CryptonQQQ commented 5 years ago

I have the same problem. Can you give me the solution to the problem？

Did you solve it?

yolo-612 commented 5 years ago

@jonghwaC no ，no one told me how to solve during this time. Is your result similar to my result? for eth_log and client_result?

jonghwaC commented 5 years ago

I have a similar result. But I want is throughput and latency.

So I am in small bank code analysis. On line 175 in smallbank.cc, there is a portion that calculates throughput and latency.

"threads.emplate_back (StatusThread, sb, props [" dbname "], props [" endpoint "] ..." Commenting out the above code shows Throughput but not Latency.

CryptonQQQ commented 5 years ago

I have a similar result. But I want is throughput and latency.

So I am in small bank code analysis. On line 175 in smallbank.cc, there is a portion that calculates throughput and latency.

"threads.emplate_back (StatusThread, sb, props [" dbname "], props [" endpoint "] ..." Commenting out the above code shows Throughput but not Latency.

I am doing the same experiment as you, but there is no result, can we communicate?

yolo-612 commented 5 years ago

@jonghwaC @CryptonQQQ I got some of result of latency has been 0. you can get the throughput and latency of ycsb?

jonghwaC commented 5 years ago

@keruisiya @CryptonQQQ I also have a latency value of 0 in smallbank atomic latency (0); is defined and the variable is not used elsewhere.

I think I should use the latency defined in StatusThread.

My plan is to terminate StatusThread when all ClientThreads are terminated. StatusThread will be modified to increase the globally defined latency value.

yolo-612 commented 5 years ago

@jonghwaC ok！thanks! If you get the right results, please let me know！ For this benchmark, I also have some doubts，hope can communicate with you， you can contact me at keruisiya0818@163.com

haochenpan commented 5 years ago

Hi, @keruisiya , @jonghwaC

Our team found this

https://github.com/ooibc88/blockbench/blob/f4563c49def905d6fadba9c031a6baf2187db6fe/src/macro/kvstore/aggregate.py.

Have you checked that and did that help?

Thanks, Haochen

listenfound commented 5 years ago

This document is not used to analyze the results. It's a document in ycsb. The author modified ycsb, so it can't be used to analyze the results.

haochenpan commented 5 years ago

Hi @listenfound ,

You are right. see #49 .

After all, I don't think their way of calculating latency makes sense. See the discussion in #64. There could be a finer way of doing this.

Thank you for your discussion.

Best wishes, Roger