Pin 3.0 Compilation

[jh46537]

Hi all,

I am trying to compile zsim against Pin 3.0 which was released about a week ago (March 24th, 2016). It appears they removed the functions GetVmLock() and ReleaseVmLock() from the Pin API.

build/opt/zsim.cpp: In function ‘LEVEL_BASE::VOID FFThread(LEVEL_BASE::VOID*)’: build/opt/zsim.cpp:1352:23: error: ‘GetVmLock’ was not declared in this scope build/opt/zsim.cpp:1355:27: error: ‘ReleaseVmLock’ was not declared in this scope scons: *** [build/opt/zsim.os] Error 1

Anyway to get zsim to work without those two calls? Thanks!

[s5z]

It used to be necessary to call these when reinstrumenting, which we do when exiting/enering fast-forwarding. Have you tried taking out these calls and seeing what happens?

[dzhang50]

I tried to get Pin 3.0 up and running but there have been some fairly significant changes. Here are some of the new changes. Can someone with more Pin expertise help get it up and running?

1. GetVmLock() and ReleaseVMLock() are deprecated. I removed the calls.
2. libdwarf.a in SConstruct is now intel64/lib-ext/libpin3dwarf.so. This is easy to fix.
3. PIN_SpawnInternalThread semantics have changed; the -tool_exit_timeout switch is now missing. Can this be safely removed? I just commented it out.
4. ZSim calls pinbin directly rather than going through the pin.sh wrapper. However, in pin 3.0, pin.sh has been removed (replaced with a pin binary) and there are linking errors when attempting to directly call pinbin. This seems to be the main problem, assuming that 1 and 3 are not an issue.

[s5z]

1. OK
2. Yes, please link against that.
3. What does the "it" in "commented it out" refer to? If you are commenting out tool_exit_timeout, that should be fine. But commenting out PIN_SpawnInternalThread will break several things.
4. Not sure how calling the binary can cause linking errors. Maybe runtime errors? In that case, you should figure out what the new format is and tweak pin_cmd.cpp.

[gaomy3832]

After looking into this problem myself for a while, I finally conclude that this may be quite a pain to make zsim support Pin 3.0, and I give up for now until there is more information from experts.

The first 3 issues above seem easy to solve. The real pain is the last one. Even after fixing the lib path (Pin now uses another env var PIN_VM64_LD_LIBRARY_PATH rather than LD_LIBRARY_PATH to locate the shared libs on which pintool depends, and it seems using -rpath doesn't work any more), the Pin launcher doesn't recognize those 3rd party libs with the error:

E: Unable to load /armadillo/users/mgao12/misc/myPinTools/tools/obj-intel64/sample_tool.so: dlopen failed: empty/missing DT_HASH in "libconfig++.so.9" (built with --hash-style=gnu?)

The biggest change of Pin 3.0 is the use of a new runtime layer, PinCRT. An overview doc can be found here. It replaces libc.so for Linux and tries to make the runtime OS-independent. As a result, several strong requirements now are:

1. No OS-specific functions can be used in pintool (pp8 of the doc). I believe zsim has used quite a few Linux-specific calls. So some effort to reconstruct the code is needed here.
2. Pintool and the depended 3rd libs (libconfig++ etc.) must all be rebuilt with PinCRT to get rid of libc.so. The rebuild should resolve the error I encountered above, but is painful to do for all libs.
3. No support for boost or C++11, which is now fine but can be a potential issue in the future.
4. Could be more issues that I haven't found.

[cyliustack]

dear akgun1, i thought pin-2.14 (seen in the following link) may be helpful for you, https://drive.google.com/file/d/0BwfK93g2BHz7d2RLWF9xTncwNUE/view?usp=sharing

2017-01-27 7:46 GMT+08:00 akgun1 notifications@github.com:

I am having the same issue right now, wanted to try out zsim for the first time. If zsim will not provide support for Pin 3.0 can someone link a Pin distribution that works? Official website for Pin only has the latest version up for download. Thanks.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/s5z/zsim/issues/109#issuecomment-275549963, or mute the thread https://github.com/notifications/unsubscribe-auth/APX2RX1nVTvEIRgzdxxEsHgXQ1tR0ZIGks5rWTBLgaJpZM4H-Kk5 .

--

• C.Y. Liu, (劉政岳) CSIE, National Taiwan University*

[akgun1]

@cyliustack Thanks for your help. Unfortunately, I get the following error when running Pin 2.14: " E: 4.4 is not a supported linux release " I'm on Ubuntu 14.04 and running 4.4 Linux kernel. I am looking for a workaround right now, please let me know if you have any suggestions.

[akgun1]

Update:

Apparently you can get Pin 2.14 run on 4.4 kernel by adding "-injection child" argument. I added an alias to bash as follows, so now Pin 2.14 can run standalone: alias pin="pin -injection child"

However I still get the following output when I try to run zsim with simple.cfg:

[H] Starting zsim, built Thu Jan 26 15:50:53 PST 2017 (rev master:89:fb4d6e0:clean)
[H] Creating global segment, 1024 MBs
[H] Global segment shmid = 12550151
[H] Deadlock detection ON
E: 4.4 is not a supported linux release
E: 4.4 is not a supported linux release
[H] Child 4335 done
[H] Child 4336 done
[H] All children done, exiting

Any help would be appreciated!

[gaomy3832]

We used to believe pin 2.x does not work on Linux kernel version 4 and later, thus the failure of E: 4.4 is not a supported linux release. Previous Ubuntu 14.04 has kernel version 3.x, and zsim works fine on it.

I am not awared of the -injection child argument of pin (I am not a pin expert). It seems to be a workaround for the issue above based on your experience. Could you provide more information on how you found this and how it works? If this is the correct way, you can modify the pin invoke command in the src file pin_cmd.cpp to pass it to pin binary in the function PinCmd::PinCmd(). Zsim uses pinbin as the binary, thus your bash alias for pin didn't work.

[Dragonslair5]

I am using the -injection child to run ZSIM on newer kernels with PIN 2.14. To do so I modified pin_bin.cpp to add this parameter to the pin call, just like @gaomy3832 suggested, with the following two lines of code right after adding the pinbin path (line 52 from pin_bin.cpp):

   `args.push_back("-injection");
    args.push_back("child");`

Apparently, according to the pin's manual, there is nothing against doing it.

[gaomy3832]

@Dragonslair5 Thanks! Good to know this trick works with zsim. I have also seen the manual but there is not too much information. Could you share any insight why it works or how did you come up with it?

[Dragonslair5]

I found it out unintentionally, while using a modified version of ZSIM to use PinPoints. This modified version was running in newer kernels just because the -injection child parameter was added.

Maybe there is a problem using parent injection in newer kernels and you need to explicit use the injection child, but I really don't know and I would also appreciate if someone with more knowledge in PIN could clarify why it works.

[hoangt]

Anybody can let me know the location of "pin_bin.cpp" file ?

[BingBingElf]

Anybody can let me know the location of "pin_bin.cpp" file ?

Do you find the right place where the "pin_bin.cpp" in? I faced with the same question. Ths!

[BingBingElf]

我在无意中发现它，同时使用修改版的ZSIM来使用PinPoints。由于添加了-injection子参数，因此此修改版本在较新的内核中运行。

也许在较新的内核中使用父注射存在问题，并且您需要明确使用注射子，但我真的不知道，如果有更多PIN知识的人可以澄清其工作原理我也会感激。

I added "-injection & child" into bin_cmd.cpp file, but there's something wrong that I cannot fix. What should I do next? Would you give me some advice? THS!

detail: -injection child /home/HSCC/zsim-nvmain/pin_kit/intel64/bin/pinbin -follow_execv -tool_exit_timeout 1 -t /home/HSCC/zsim-nvmain/bin/libzsim.so -config /home/HSCC/zsim-nvmain/config/shma-static.cfg -outputDir /home/HSCC/zsim-nvmain -shmid 7176205 -logToFile -procIdx 0 -- /473.astar/exe/astar_base.amd64-m64-gcc43-nn BigLakes2048.cfg Could not exec, killing child: No such file or directory [H] Panic on build/opt/zsim_harness.cpp:300: Could not exec -injection

[Dragonslair5]

The file is actually pin_cmd.cpp.

The trick is, instead of doing

 //Global pin options
 args.push_back("-follow_execv"); //instrument child processes
 args.push_back("-tool_exit_timeout"); //don't wait much of internal threads
 args.push_back("1");

You do

 args.push_back("-injection");
 args.push_back("child");
 args.push_back("-follow_execv"); //instrument child processes
 args.push_back("-tool_exit_timeout"); //don't wait much of internal threads
 args.push_back("1");

There is a flag that I knew later, ifeellucky, that also seems to let you use older PIN on newer kernels, that is

args.push_back("-ifeellucky");

There is also a fork from this repository that seems to have given a solution to run ZSIM on newer kernels, that is similar to my solution.

Just note that these are solutions to run an older version of PIN on newer kernels, instead of using PIN 3.0+ as the title of this thread suggests.

[BingBingElf]

The file is actually pin_cmd.cpp.

The trick is, instead of doing

 //Global pin options
 args.push_back("-follow_execv"); //instrument child processes
 args.push_back("-tool_exit_timeout"); //don't wait much of internal threads
 args.push_back("1");

You do

 args.push_back("-injection");
 args.push_back("child");
 args.push_back("-follow_execv"); //instrument child processes
 args.push_back("-tool_exit_timeout"); //don't wait much of internal threads
 args.push_back("1");

There is a flag that I knew later, ifeellucky, that also seems to let you use older PIN on newer kernels, that is

args.push_back("-ifeellucky");

There is also a fork from this repository that seems to have given a solution to run ZSIM on newer kernels, that is similar to my solution.

Just note that these are solutions to run an older version of PIN on newer kernels, instead of using PIN 3.0+ as the title of this thread suggests.

Thank you very much for your answer. Trying again will still give an error, so I tried to test it on kernel version 3.X, and the result is fine. Thanks again!

[qianlong-zhang]

Above workaround will case zsim debugger fail, ie. attachDebugger in zsim config file will not invoke gdb and Xterm anymore if args.push_back("child"); is used. Because pintool will create a new process id if args.push_back("child"); is used. Instead, args.push_back("parent"); should be used, then the notifyHarnessForDebugger can send signal to right parent(harness process), or the signal will be sent to the child of harness which cause harness process miss the debug signal.

[ericwu13]

Hi all,

I am trying to compile zsim against Pin 3.0 which was released about a week ago (March 24th, 2016). It appears they removed the functions GetVmLock() and ReleaseVmLock() from the Pin API.

build/opt/zsim.cpp: In function ‘LEVEL_BASE::VOID FFThread(LEVEL_BASE::VOID*)’: build/opt/zsim.cpp:1352:23: error: ‘GetVmLock’ was not declared in this scope build/opt/zsim.cpp:1355:27: error: ‘ReleaseVmLock’ was not declared in this scope scons: *** [build/opt/zsim.os] Error 1

Anyway to get zsim to work without those two calls? Thanks!

Hi,

I'm trying to build zsim with pin3.11 on ubuntu 14.04 (kernel 4.4, g++ 4.8). I wonder if you success with this? I don't know how to revise my SConstruct file to build zsim with pincrt properly. I tried to include the /extra/crt/include and others related directories suggessted by the PinCRT manual. But still got some error. Hope you can help me. Thanks.

Sincerely, Eric

[YalandHong]

Above workaround will case zsim debugger fail, ie. attachDebugger in zsim config file will not invoke gdb and Xterm anymore if args.push_back("child"); is used. Because pintool will create a new process id if args.push_back("child"); is used. Instead, args.push_back("parent"); should be used, then the notifyHarnessForDebugger can send signal to right parent(harness process), or the signal will be sent to the child of harness which cause harness process miss the debug signal.

@qianlong-zhang Thanks, Zhang! Your answer solved a problem that had bewildered and frustrated me for several days.

Worse still, the PinTool seems not able to handle the signal (SIGUSR1), which will immediately crash the whole program. I tried it on my Ubuntu 14.04 and when I set attachDebugger to true, the simulator crashed during SimInit.

[Aileen-Huang]

dear akgun1, i thought pin-2.14 (seen in the following link) may be helpful for you, https://drive.google.com/file/d/0BwfK93g2BHz7d2RLWF9xTncwNUE/view?usp=sharing 2017-01-27 7:46 GMT+08:00 akgun1 notifications@github.com: … I am having the same issue right now, wanted to try out zsim for the first time. If zsim will not provide support for Pin 3.0 can someone link a Pin distribution that works? Official website for Pin only has the latest version up for download. Thanks. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#109 (comment)>, or mute the thread https://github.com/notifications/unsubscribe-auth/APX2RX1nVTvEIRgzdxxEsHgXQ1tR0ZIGks5rWTBLgaJpZM4H-Kk5 . -- C.Y. Liu, (劉政岳) CSIE, National Taiwan University

sorry it need access, could you please share again?

[YalandHong]

dear akgun1, i thought pin-2.14 (seen in the following link) may be helpful for you, https://drive.google.com/file/d/0BwfK93g2BHz7d2RLWF9xTncwNUE/view?usp=sharing 2017-01-27 7:46 GMT+08:00 akgun1 notifications@github.com: … I am having the same issue right now, wanted to try out zsim for the first time. If zsim will not provide support for Pin 3.0 can someone link a Pin distribution that works? Official website for Pin only has the latest version up for download. Thanks. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#109 (comment)>, or mute the thread https://github.com/notifications/unsubscribe-auth/APX2RX1nVTvEIRgzdxxEsHgXQ1tR0ZIGks5rWTBLgaJpZM4H-Kk5 . -- C.Y. Liu, (劉政岳) CSIE, National Taiwan University

sorry it need access, could you please share again?

Intel still provides an official link to Pin 2.14: https://software.intel.com/sites/landingpage/pintool/downloads/pin-2.14-71313-gcc.4.4.7-linux.tar.gz

Besides, some GitHub users have uploaded their own copies of Pin 2.13/2.14. You may find them using Google or GitHub search.

[maitpate]

hi I am trying to use pin-2.14-71313-gcc.4.4.7-linux , getting below error on execution : pin-2.14-71313-gcc.4.4.7-linux/pin: cannot execute binary file: Exec format error any idea on this issue ?