search

clangupc / clang-upc

Clang UPC Front-End
https://clangupc.github.io/
Other
16 stars 5 forks source link

RFE: port UPC runtime to {Free,Net,Open}BSD #54

Open PHHargrove opened 10 years ago

PHHargrove commented 10 years ago

I have built and successfully tested clang-upc2c on recent versions of FreeBSD, NetBSD and OpeBSD. I then tried clang-upc as well and was unable to get a working system.

My initial attempts to get clang-upc running on the BSD systems was not passing the necessary options to the linker to get the UPC runtime. I have addressed the link issues for all three systems at https://github.com/PHHargrove/clang-upc/tree/bsd-support

In addition to the changes in my bsd-support branch, one must pass -DLIBUPC_ENABLE_BACKTRACE=FALSE to cmake on these systems).

Having gotten past the linker issues on all three systems, I now find that on all three systems the UPC runtime aborts at startup for even a simple "Hello, World!" UPC program:

$ ./a.out -n1
./a.out: UPC initialization failed.
./a.out: reason: Invalid argument
Abort trap (core dumped)

So, this issue is an RFE for porting of the UPC runtime to BSD systems. I have no clue at this point in time as to the amount of work that may be required.

nenadv commented 10 years ago

I tried making changes for freebsd, just cut and paste from gnu tools. But, was not able to link it as upc.ld/upc-crt files, etc.. were not found. Freebsd places "-Tupc.ld" on the command line, while Linux places "-Tfull-path/upc.ld" and is able to find it. The same with upc-crtbegin/end files.

After debugging I can see that freebsd searches only two paths: ../lib/clang/34 and /usr/lib. Our upc.ld, upc-begin/end are installed in ../lib directory.

nenadv commented 10 years ago

Paul, it looks like I did not make the change to ToolChains.cpp that adds "../lib" to the driver paths. I wonder if this is the FreeBsd bug (as Linux, and others have it) or something specific to that system. Maybe we are placing UPC related stuff in the wrong directory, ../lib instead of ../lib/clang/3.4.

PHHargrove commented 10 years ago

Nenab,

It sounds like you are trying to make the same changes I already made on the branch I reference in the initial post. With the changes I made I am able to link just fine, which is why I shared that branch. Was there something wrong with the changes I made that is motivating you to try something different?

-Paul

nenadv commented 10 years ago

Yes, I did start with your changes but missed the file that adds 'lib' on the path. I guess I was looking at the openbsd page.

Can you update the master with your changes? I'll make few changes after that to fix the backtrace link issue. Also, the cause for the run failure is the 'fd' argument on the mmap call that has to be '-1' for freebsd. We have it at zero and it seems that linux does not care. Any experience with that? We use anonymous memory map.

PHHargrove commented 10 years ago

Nenad,

I will push my changes to 'master' soon. I didn't do so initially because without a working runtime they didn't seem to serve any actual purpose.

As for anonymous mmap: GASNet uses '-1' with MAP_ANON(YMOUS).

FYI: I quickly verified that OpenBSD, NetBSD and FreeBSD manpages for mmap() all agree that fd must be -1 for anonymous mappings.

If it would be helpful, I can provide access to our NetBSD and OpenBSD systems. They are actual h/w now, but will migrate to VMs this summer.

-Paul

PHHargrove commented 10 years ago

My 'bsd-support' branch has been merged to master and the branch deleted.

PHHargrove commented 10 years ago

By following Nenad's suggestion and just changing the fd argument to mmap() from 0 to -1, I am able to run hello.upc on FreeBSD-10/amd64. So, is there any objection to me committing the one-line change to fd==-1 in the smp runtime?

What about the mmap(ANON) call in the portals4 runtime? While that is Linux-only right now I see no reason not to make the change there for future portability.

I am going to get at least the intrepid suite tested on FreeBSD-10 and OpenBSD-5 on amd64. My NetBSD-6/am64 platform is down due to h/w issues but I think I have a suitable VM. I will report back on the results later.

-Paul

PHHargrove commented 10 years ago

At least on FreeBSD I am getting link errors on some tests because the ld command line is placing -lupc before the .o file(s) rather than after. Swapping the order fixes that.

Look for a commit to master resolving that sometime soon. Corresponding changes for OpenBSD and NetBSD will follow if necessary.

nenadv commented 10 years ago

I made the last change for freebsd runtime, making sure that we don't brake linux - https://github.com/Intrepid/clang-upc/commit/364debfd1e3181c6017d25abddbd745d34b6097c

PHHargrove commented 10 years ago

Using a local mod of the same fd=-1 change that Nenad just committed, I was able to get the intrepid suite to pass on FreeBSD-10, NetBSD-6 and OpenBSD-5, all on amd64. I saw one fail on test29 due to some fp imprecission issue.

I have full harness runs in-flight now.

PHHargrove commented 10 years ago

Full harness runs on {Free,Net,Open}BSD/amd64 revealed no new failures attributable to the runtime. I have a few follow-up commits to the testsuite, but otherwise we seem to have a successful triple-port!

I have looked at Solaris, but it has additional issues that are unrelated to the runtime. So, I am going to declare this issue RESOLVED, and open a separate one for a Solaris port.

PHHargrove commented 10 years ago

Lacking any other equally-visible place for these notes:

nenadv commented 10 years ago

Ha, almost forgot backtraceupdate. Paul, can you confirm that "/usr/lib/libexecinfo*" is also available on openbsd and netbsd. That is their version of backtrace functionality since version 6.

On 6/24/14 8:16 PM, Paul H. Hargrove wrote:

Lacking any other equally-visible place for these notes:

  • FreeBSD-10.0/amd64 and NetBSD-6.1/amd64 o Pass |-DLIBUPC_ENABLE_BACKTRACE=FALSE| to cmake o Needed |-gdwarf-2| on clang-upc (and clang) command line to work w/ system's linker
  • OpenBSD-5.5/amd64 o Pass |-DLIBUPC_ENABLE_BACKTRACE=FALSE| to cmake o Needed |-g0| on clang-upc (and clang) command line to work w/ system's linker

— Reply to this email directly or view it on GitHub https://github.com/Intrepid/clang-upc/issues/54#issuecomment-47056523.

PHHargrove commented 10 years ago

Summary of backtrace support (libexecinfo) in the BSD variants:

All three have both backtrace() and backtrace_symbols() when they have libexecinfo at all.

nenadv commented 10 years ago

Made additional update to add libexecinfo on the UPC linker line on BSD systems - https://github.com/Intrepid/clang-upc/commit/d4601bab3869d2e6734f7bbcf52c32cb434b5865. My testing on freebsd shows that cmake also depends in this library so we have a good chance of the library being there.

This patch might not be optimal as it completely disables backtrace, including the backtrace on segfaults which can be done with GDB if available. We probably need to rearrange the code and separate aborts from trace on demand.

PHHargrove commented 10 years ago

Re backtrace: one success and two failures.

Summary:

So, I still need to disable backtrace support manually on NetBSD and OpenBSD

FreeBSD

My FreeBSD has libexecinfo by default and it appears I can get backtrace support correctly linked:

$ ./rls/bin/clang-upc ~/hello.upc   
$ nm a.out |grep backtrace
0000000000408310 T __upc_backtrace
0000000000408e90 t __upc_backtrace_handler
0000000000408b60 T __upc_backtrace_init
0000000000408fc0 t __upc_backtrace_monitor
0000000000408f30 T __upc_backtrace_restore_handlers
00000000004088c0 T __upc_fatal_backtrace
                 U backtrace
                 U backtrace_symbols
$ ./a.out -n2
 - Hello from thread 0
 - Hello from thread 1

But with a program that derefs NULL I don't see a backtrace:

$ ./rls/bin/clang-upc -g ~/segv.upc 
$ ./a.out -n1
Segmentation fault (core dumped)

However, I then found I couldn't get a backtrace on Linux or Darwin either. So, I assume I am missing some setting/switch.

NetBSD

On NetBSD I have the optional libexecinfo package installed, but the symbol was not found:

-bash-4.2$ grep backtrace logs/cmake.log 
-- Looking for backtrace
-- Looking for backtrace - not found
-- Looking for backtrace in execinfo
-- Looking for backtrace in execinfo - not found

I don't know if I am expected to set LD_LIBRARY_PATH to get access to libs in /usr/pkg, but regardless of that the CMake logic appears broken and does not result in automatically disabling backtrace in libupc as was apparently intended:

$ ./rls/bin/clang-upc ~/hello.upc 
/usr/bin/ld: cannot find -lexecinfo
clang-3.4: error: linker command failed with exit code 1 (use -v to see invocation)

This was starting from an empty bld directory and therefore cannot be a missing dependency.

OpenBSD

On OpenBSD I have the optional libexecinfo package installed and the symbol is found in libexecinfo, though I have not set LD_LIBRARY_PATH.

$ grep backtrace logs/cmake.log
-- Looking for backtrace
-- Looking for backtrace - not found
-- Looking for backtrace in execinfo
-- Looking for backtrace in execinfo - found

However, since execinfo is an optional package, the header is located under /usr/local/include. This leads to a build failure in upc_backtrace.c:

[ 90%] Building C object tools/clang/runtime/libupc/CMakeFiles/upc-s-l.dir/smp/upc_backtrace.c.o
/home/phargrov/llvm-upc/src/llvm/tools/clang/runtime/libupc/smp/upc_backtrace.c:20:10: fatal error: 
      'execinfo.h' file not found
#include <execinfo.h>
         ^
1 error generated.
nenadv commented 10 years ago

To get backtraces you need to set "UPC_BACKTRACE=1" env variable.

I guess for now you should disable backtrace. I was afraid that more configuration checks would be required.

I would like to check on the NetBSD - not found but still adds execinfo on the library path. I did try to test it on freebsd but cmake required libexecinfo in order to run.

On 6/25/2014 4:05 PM, Paul H. Hargrove wrote:

Re backtrace: one success and two failures.

  Summary:
  • FreeBSD builds and links fine, but I don't know how to test backtraces.
  • NetBSD fails to find backtrace in libexecinfo but fails to link executables
  • OpenBSD finds backtrace in libececinfo but fails to build libupc

So, I still need to disable backtrace support manually on NetBSD and OpenBSD

  FreeBSD

My FreeBSD has libexecinfo by default and it appears I can get backtrace support correctly linked:

|$ ./rls/bin/clang-upc ~/hello.upc $ nm a.out |grep backtrace 0000000000408310 T upc_backtrace 0000000000408e90 t upc_backtrace_handler 0000000000408b60 T upc_backtrace_init 0000000000408fc0 t upc_backtrace_monitor 0000000000408f30 T upc_backtrace_restore_handlers 00000000004088c0 T upc_fatal_backtrace U backtrace U backtrace_symbols $ ./a.out -n2

  • Hello from thread 0
  • Hello from thread 1 |

But with a program that derefs NULL I don't see a backtrace:

$ ./rls/bin/clang-upc -g ~/segv.upc $ ./a.out -n1 Segmentation fault (core dumped)

However, I then found I couldn't get a backtrace on Linux or Darwin either. So, I assume I am missing some setting/switch.

  NetBSD

On NetBSD I have the optional libexecinfo package installed, but the symbol was not found:

-bash-4.2$ grep backtrace logs/cmake.log -- Looking for backtrace -- Looking for backtrace - not found -- Looking for backtrace in execinfo -- Looking for backtrace in execinfo - not found

I don't know if I am expected to set LD_LIBRARY_PATH to get access to libs in /usr/pkg, but regardless of that the CMake logic appears broken and does not result in automatically disabling backtrace in libupc as was apparently intended:

$ ./rls/bin/clang-upc ~/hello.upc /usr/bin/ld: cannot find -lexecinfo clang-3.4: error: linker command failed with exit code 1 (use -v to see invocation)

This was starting from an empty bld directory and therefore cannot be a missing dependency.

  OpenBSD

On OpenBSD I have the optional libexecinfo package installed and the symbol /is/ found in libexecinfo, though I have not set LD_LIBRARY_PATH.

$ grep backtrace logs/cmake.log -- Looking for backtrace -- Looking for backtrace - not found -- Looking for backtrace in execinfo -- Looking for backtrace in execinfo - found

However, since execinfo is an optional package, the header is located under /usr/local/include. This leads to a build failure in |upc_backtrace.c|:

|[ 90%] Building C object tools/clang/runtime/libupc/CMakeFiles/upc-s-l.dir/smp/upc_backtrace.c.o /home/phargrov/llvm-upc/src/llvm/tools/clang/runtime/libupc/smp/upc_backtrace.c:20:10: fatal error: 'execinfo.h' file not found

include 

      ^

1 error generated. |

— Reply to this email directly or view it on GitHub https://github.com/Intrepid/clang-upc/issues/54#issuecomment-47170284.

PHHargrove commented 10 years ago

I would like to check on the NetBSD

I think it is a CMake syntax issue. You have

  if (NOT ${HAVE_EXECINFO_LIB})

where I think 

  if (NOT HAVE_EXECINFO_LIB)

is required.

I am testing that change now and will commit if it works.

Some CMake docs suggest that find_library and find_path might be the right tools to locate the proper -L and -I options.

nenadv commented 10 years ago

The same code is also in runtime/libupc/CMakeList.txt

On 6/25/2014 4:25 PM, Paul H. Hargrove wrote:

I would like to check on the NetBSD

I think it is a CMake syntax issue. You have

if (NOT ${HAVE_EXECINFO_LIB})

where I think

if (NOT HAVE_EXECINFO_LIB)

is required.

I am testing that change now and will commit if it works.

Some CMake docs http://www.cmake.org/Wiki/CMake:How_To_Find_Libraries suggest that |find_library| and |find_path| might be the right tools to locate the proper |-L| and |-I| options.

— Reply to this email directly or view it on GitHub https://github.com/Intrepid/clang-upc/issues/54#issuecomment-47171629.

PHHargrove commented 10 years ago

I've confirmed that removing curly braces from the two instances of if (NOT ${HAVE_EXECINFO_LIB}) allows NetBSD to automatically disable backtrace support as expected. This has been pushed as 30b2f15.

For the moment there are still issues to resolve with backtrace support

So, the bug report remains open.

PHHargrove commented 10 years ago

x86 (32-bit) builds on FreeBSD and NetBSD have passed the Intrepid suite "smoke test".

OpenBSD has problems (both 32-bit and 64-bit) related to the linker barfing on clang's Dwarf output on many tests. However, I can build hello.upc just fine. That is not a UPC-related issue as far as I can tell.

I now have weekly runs of both 32- and 64-bit clang-upc on FreeBSD-10. I expect to add NetBSD as well.

nenadv commented 10 years ago

I created a "freebsd" branch with cmake changes for NetBSD and OpenBSD UPC backtracing. If this works I'll add configure/Makefile part.

PHHargrove commented 10 years ago

I will test the "freebsd" branch when I can - hopefully by tonight.

PHHargrove commented 10 years ago

Results of testing "freebsd" branch (at c281c64) with CMake:

So, nothing got worse and OpenBSD got better: automatically disqualifies backtrace support rather than failing while building libupc unless backtrace was manually disabled.

Nenad,

If this new code was expected to successfully locate backtrace support on OpenBSD or NetBSD, then it has failed in that respect. Let me know if you need any files from me, or if you want access to my systems.

nenadv commented 10 years ago

At this point the best is to get access and keep you out of the testing duties.

On 6/26/14 5:03 PM, Paul H. Hargrove wrote:

Results of testing "freebsd" branch (at c281c64 https://github.com/Intrepid/clang-upc/commit/c281c64) with CMake:

  • FreeBSD: no change -- still works by default
  • NetBSD: no change -- still fails to locate any backtrace support
  • OpenBSD: POSITIVE change -- now builds OK but still without backtrace support

So, nothing got worse and OpenBSD got better: automatically disqualifies backtrace support rather than failing while building libupc unless backtrace was manually disabled.

Nenad,

If this new code was expected to successfully locate backtrace support on OpenBSD or NetBSD, then it has failed in that respect. Let me know if you need any files from me, or if you want access to my systems.

— Reply to this email directly or view it on GitHub https://github.com/Intrepid/clang-upc/issues/54#issuecomment-47294793.