How to reproduce the SimPoint in the paper?

[Gerrie-Cui]

Hello! I have read your paper. Saw the introduction about SimPoint. And your final result. But I am still confused about how to generate my own trace. How can I reproduce the results in your paper?

My current understanding is that the steps to generate SimPoint are:

1. First generate BBV.
2. Then use the weight of each cluster that generated it and the starting instruction position.

The method of generating BBV, my Google result is to use valgrind --tool=exp-bbv execute_file. Use SimPoint to complete the second step.

I want to know how you do it? Can you introduce your process with an example? Can you introduce your process with an example? E.g. 500.perbench_s

Thank you so much!

[BamboWu]

Your understanding is correct. For step1 generating BBV, valgrind works, while we used Intel PinTool in order to generate the checkpoints that later can be used in SniperSim. For examples, you may want to refer to PinPoints and PinPlay pages.

[Gerrie-Cui]

I want to generate SimPoint for ChampSim. But I found that ChampSim did not use parameters when running the benchmark program in spec2017. ./bin/${BINARY} -warmup_instructions ${N_WARM}000000 -simulation_instructions ${N_SIM}000000 ${OPTION} -traces ${TRACE_DIR}/${TRACE0} ${TRACE_DIR}/${TRACE1} ${TRACE_DIR}/${TRACE2} ${TRACE_DIR}/${TRACE3}) &> ${RESULT_DIR}/mix${N_MIX}-${BINARY}${OPTION}_${N_SIM}M.txt

When using runcpu to run the benchmark in spec2017. There will be some parameters. For example, when running perlbench, the command line is: ./perlbench -I. -I./lib suns.pl > suns.out 2>> suns.err

Is this parameter set when using Pintool to generate trace? But I did not find the corresponding option for setting parameters? Thanks

[BamboWu]

Pintool could run benchmark with parameters. Just append the parameters after the benchmark. For example:

$ ../../../pin -t obj-intel64/inscount0.so -o inscount0.log -- /path/to/spec/600.perlbench_s/perlbench -I/path/to/spec/600.perlbench_s/ -I/path/to/spec/600.perlbench_s/lib /path/to/spec/600.perlbench_s/suns.pl

[Gerrie-Cui]

Thank you very, very much for your answer!

I'm still a bit confused. I first used valgrind --tool=exp-bbv ./perlbench_s_base.amd64-m64 to generate bbv files. This lasted all day without ending.

My main confusion lies in:

1. When generating the bbv file, should I also enter the parameters? Different parameters correspond to different program execution processes. For example, the input parameters may determine whether the branch is taken. The introduction to BBV in your paper is: "Counting how many times each basic block is executed during a region’s execution produces a vector known as Basic Block Vector (BBV)". Then the execution of the region is related to the parameters.

Is the command to generate BBV

valgrind --tool=exp-bbv  /path/to/spec/600.perlbench_s/perlbench -I/path/to/spec/600. perlbench_s/ -I/path/to/spec/600.perlbench_s/lib /path/to/spec/600.perlbench_s/suns.pl?

But when I execute valgrind --tool=exp-bbv ./perlbench_s_base.amd64-m64.bak -I./lib checkspam.pl 2500 5 25 11 150 1 1 1 1, the following error occurred.

==6979== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
==6979== Command: ./perlbench_s_base.amd64-m64 -I./lib checkspam.pl 2500 5 25 11 150 1 1 1 1
==6979==
==6979== brk segment overflow in thread #1: can't grow to 0x4a48000
==6979== brk segment overflow in thread #1: can't grow to 0x4a4a000
==6979== brk segment overflow in thread #1: can't grow to 0x4a48000
==6979== brk segment overflow in thread #1: can't grow to 0x4a4f000
==6979== brk segment overflow in thread #1: can't grow to 0x4a49000
==6979== brk segment overflow in thread #1: can't grow to 0x4a4a000

But I am directly run ./perlbench_s_base.amd64-m64 -I./lib diffmail.pl 4 800 10 17 19 300, it's no errors.

Is it convenient for you to disclose the commands you use to generate BBV? I also currently have PinTool installed. But I don't know how to use it to generate BBV files.

2. There are several sets of parameters for each benchmark. Are the corresponding situations of these three sets of parameters similar? I plan to do some prefetch experiments, so it is enough to simulate 100 million instructions on the simulator. Can I just use a set of parameters?

3. ./perlbench -I./lib checkspam.pl 2500 5 25 11 150 1 1 1 1 > checkspam .2500.5.25.11.150.1.1.1.1.out 2>> checkspam.2500.5.25.11.150.1.1.1.1.err 2 ./perlbench -I./lib diffmail.pl 4 800 10 17 19 300 > diffmail.4.800.10.17.19.300.out 2>> diffmail.4.800.10.17.19.300.err 3 ./perlbench -I./lib splitmail.pl 6400 12 26 16 100 0 > splitmail.6400.12.26.16.100.0. out 2>> splitmail.6400.12.26.16.100.0.err

   
   Thank you very much! This helped me a lot.

[BamboWu]

For 1. Yes, you do need the parameters. The valgrind command you provided works fine on our server (65 GB). Please check whether valgrind gets enough memory it needs on your machine. If you want to try the same approach as what we did, you need to download PinPlay, and use the sniper_pinpoints.py script. There would be a bunch of commands to set it up, so you would better refer to its documents and tutorials (e.g., 2016 PLDI).

For 2. You may want to pick the representative one from the set of parameters. Please refer to TABLE VII in Wait of a Decade: Did SPEC CPU 2017 Broaden the Performance Horizon?.

[Gerrie-Cui]

Thank you very much for your answers! That helped me a lot.