RFE: port to Solaris

[PHHargrove]

This issue pretty much just a place holder until I can get time to resume the work I've started (but already forgotten the details of).

The first stumbling block I hit was that the Solaris platform is not using a GNU toolchain.
So, at a minimum we need to disable the linker script on Solaris.

The clang infrastructure knows how to use the Solaris linker appears to work on it own (w/o upc). So, hopefully the work required will be relatively small.

I am only targeting x86-64 at the moment.

[PHHargrove]

With commit ef69081 we have sufficient Solaris-11 support to compile and run at least the intrepid tests on x86-64.

However, every single upc execution prints:

./a.out:  UPC Warning: There are no UPC source files compiled into this program, or <upc.h> was not included?

Any clue how I can fix that?

[nenadv]

Seems something is wrong with the program info section. Empty?

Can you print the file headers and check what you have in upc_pgm_info(?) section. Might be some kind of misalignment of data and our parser of the section does not see anything in it. I recall seeing the similar issue.

If you can debug it put a breakpoint at __upc_validate_pgm_info and follow the code.

On 6/25/14 3:10 AM, Paul H. Hargrove wrote:

With commit ef69081 https://github.com/Intrepid/clang-upc/commit/ef69081 we have sufficient Solaris-11 support to compile and run at least the intrepid tests on x86-64.

However, every single upc execution prints:

./a.out: UPC Warning: There are no UPC source files compiled into this program, or was not included?

Any clue how I can fix that?

— Reply to this email directly or view it on GitHub https://github.com/Intrepid/clang-upc/issues/58#issuecomment-47082984.

[PHHargrove]

Looks like there are multiple upc_pgm_info sections, one with what I assume is the expected content and one with just the zero byte (from the upc-crt code?). They are NOT contiguous.

Contents of section upc_pgm_info:
 80655c0 24474343 55504343 6f6e6669 673a2028  $GCCUPCConfig: (
 80655d0 3c627569 6c742d69 6e3e2920 64796e61  <built-in>) dyna
 80655e0 6d696374 68726561 64732070 726f6365  micthreads proce
 80655f0 73732400 00000000 00000000 00000000  ss$.............
 8065600 24474343 55504343 6f6e6669 673a2028  $GCCUPCConfig: (
 8065610 2f686f6d 652f7068 61726772 6f762f6c  /home/phargrov/l
 8065620 6c766d2d 7570632f 7372632f 6c6c766d  lvm-upc/src/llvm
 8065630 2f746f6f 6c732f63 6c616e67 2f72756e  /tools/clang/run
 8065640 74696d65 2f6c6962 7570632f 736d702f  time/libupc/smp/
 8065650 7570635f 616c6c6f 632e7570 63292064  upc_alloc.upc) d
 8065660 796e616d 69637468 72656164 73207072  ynamicthreads pr
 8065670 6f636573 73240000 00000000 00000000  ocess$..........
 8065680 24474343 55504343 6f6e6669 673a2028  $GCCUPCConfig: (
 8065690 3c627569 6c742d69 6e3e2920 64796e61  <built-in>) dyna
 80656a0 6d696374 68726561 64732070 726f6365  micthreads proce
 80656b0 73732400 00000000 00000000 00000000  ss$.............
 80656c0 24474343 55504343 6f6e6669 673a2028  $GCCUPCConfig: (
 80656d0 2f686f6d 652f7068 61726772 6f762f6c  /home/phargrov/l
 80656e0 6c766d2d 7570632f 7372632f 6c6c766d  lvm-upc/src/llvm
 80656f0 2f746f6f 6c732f63 6c616e67 2f72756e  /tools/clang/run
 8065700 74696d65 2f6c6962 7570632f 736d702f  time/libupc/smp/
 8065710 7570635f 62617272 6965722e 75706329  upc_barrier.upc)
 8065720 2064796e 616d6963 74687265 61647320   dynamicthreads
 8065730 70726f63 65737324 00000000 00000000  process$........
 8065740 24474343 55504343 6f6e6669 673a2028  $GCCUPCConfig: (
 8065750 3c627569 6c742d69 6e3e2920 64796e61  <built-in>) dyna
 8065760 6d696374 68726561 64732070 726f6365  micthreads proce
 8065770 73732400 00000000 00000000 00000000  ss$.............
 8065780 24474343 55504343 6f6e6669 673a2028  $GCCUPCConfig: (
 8065790 2f686f6d 652f7068 61726772 6f762f6c  /home/phargrov/l
 80657a0 6c766d2d 7570632f 7372632f 6c6c766d  lvm-upc/src/llvm
 80657b0 2f746f6f 6c732f63 6c616e67 2f72756e  /tools/clang/run
 80657c0 74696d65 2f6c6962 7570632f 736d702f  time/libupc/smp/
 80657d0 7570635f 6c6f636b 2e757063 29206479  upc_lock.upc) dy
 80657e0 6e616d69 63746872 65616473 2070726f  namicthreads pro
 80657f0 63657373 24000000 00000000 00000000  cess$...........
 8065800 24474343 55504343 6f6e6669 673a2028  $GCCUPCConfig: (
 8065810 3c627569 6c742d69 6e3e2920 64796e61  <built-in>) dyna
 8065820 6d696374 68726561 64732070 726f6365  micthreads proce
 8065830 73732400 00000000 00000000 00000000  ss$.............
 8065840 24474343 55504343 6f6e6669 673a2028  $GCCUPCConfig: (
 8065850 68656c6c 6f2e7570 63292064 796e616d  hello.upc) dynam
 8065860 69637468 72656164 73207072 6f636573  icthreads proces
 8065870 732400                               s$.
[...]
Contents of section upc_pgm_info:
 8075c0c 00  

As for debugger, it seems that (as seen on the BSDs) clang is generating dwarf-4 by default. Passing -gdwarf-2 to the clang-upc command still gets dwarf errors from gdb (Cannot handle DW_FORM_<unknown>). So, I cannot yet single-step through the pgm_info validation code.

[nenadv]

This looks like a similar problem to one I encountered while debugging no link script option. Two upc_shared sections were showing up in my output, one from crt programs and the other for the rest. crt is compiled with "c", the rest with UPC.

I'll see if I can patch llvm-upc on our machine.

On 6/25/2014 12:13 PM, Paul H. Hargrove wrote:

Looks like there are multiple upc_pgm_info sections, one with what I assume is the expected content and one with just the zero byte (from the upc-crt code?). They are NOT contiguous.

Contents of section upc_pgm_info: 80655c0 24474343 55504343 6f6e6669 673a2028 $GCCUPCConfig: ( 80655d0 3c627569 6c742d69 6e3e2920 64796e61 ) dyna 80655e0 6d696374 68726561 64732070 726f6365 micthreads proce 80655f0 73732400 00000000 00000000 00000000 ss$............. 8065600 24474343 55504343 6f6e6669 673a2028 $GCCUPCConfig: ( 8065610 2f686f6d 652f7068 61726772 6f762f6c /home/phargrov/l 8065620 6c766d2d 7570632f 7372632f 6c6c766d lvm-upc/src/llvm 8065630 2f746f6f 6c732f63 6c616e67 2f72756e /tools/clang/run 8065640 74696d65 2f6c6962 7570632f 736d702f time/libupc/smp/ 8065650 7570635f 616c6c6f 632e7570 63292064 upc_alloc.upc) d 8065660 796e616d 69637468 72656164 73207072 ynamicthreads pr 8065670 6f636573 73240000 00000000 00000000 ocess$.......... 8065680 24474343 55504343 6f6e6669 673a2028 $GCCUPCConfig: ( 8065690 3c627569 6c742d69 6e3e2920 64796e61 ) dyna 80656a0 6d696374 68726561 64732070 726f6365 micthreads proce 80656b0 73732400 00000000 00000000 00000000 ss$............. 80656c0 24474343 55504343 6f6e6669 673a2028 $GCCUPCConfig: ( 80656d0 2f686f6d 652f7068 61726772 6f762f6c /home/phargrov/l 80656e0 6c766d2d 7570632f 7372632f 6c6c766d lvm-upc/src/llvm 80656f0 2f746f6f 6c732f63 6c616e67 2f72756e /tools/clang/run 8065700 74696d65 2f6c6962 7570632f 736d702f time/libupc/smp/ 8065710 7570635f 62617272 6965722e 75706329 upc_barrier.upc) 8065720 2064796e 616d6963 74687265 61647320 dynamicthreads 8065730 70726f63 65737324 00000000 00000000 process$........ 8065740 24474343 55504343 6f6e6669 673a2028 $GCCUPCConfig: ( 8065750 3c627569 6c742d69 6e3e2920 64796e61 ) dyna 8065760 6d696374 68726561 64732070 726f6365 micthreads proce 8065770 73732400 00000000 00000000 00000000 ss$............. 8065780 24474343 55504343 6f6e6669 673a2028 $GCCUPCConfig: ( 8065790 2f686f6d 652f7068 61726772 6f762f6c /home/phargrov/l 80657a0 6c766d2d 7570632f 7372632f 6c6c766d lvm-upc/src/llvm 80657b0 2f746f6f 6c732f63 6c616e67 2f72756e /tools/clang/run 80657c0 74696d65 2f6c6962 7570632f 736d702f time/libupc/smp/ 80657d0 7570635f 6c6f636b 2e757063 29206479 upc_lock.upc) dy 80657e0 6e616d69 63746872 65616473 2070726f namicthreads pro 80657f0 63657373 24000000 00000000 00000000 cess$........... 8065800 24474343 55504343 6f6e6669 673a2028 $GCCUPCConfig: ( 8065810 3c627569 6c742d69 6e3e2920 64796e61 ) dyna 8065820 6d696374 68726561 64732070 726f6365 micthreads proce 8065830 73732400 00000000 00000000 00000000 ss$............. 8065840 24474343 55504343 6f6e6669 673a2028 $GCCUPCConfig: ( 8065850 68656c6c 6f2e7570 63292064 796e616d hello.upc) dynam 8065860 69637468 72656164 73207072 6f636573 icthreads proces 8065870 732400 s$. [...] Contents of section upc_pgm_info: 8075c0c 00

As for debugger, it seems that (as seen on the BSDs) clang is generating dwarf-4 by default. Passing -gdwarf-2 to the clang-upc command still gets dwarf errors from gdb (|Cannot handle DWFORM|). So, I cannot yet single-step through the pgm_info validation code.

— Reply to this email directly or view it on GitHub https://github.com/Intrepid/clang-upc/issues/58#issuecomment-47145246.

[PHHargrove]

BTW: note that since solaris is not using the gnu linker, my build disabled the linker script. Perhaps that is related to the multiple upc_pgm_info sections?

I am not sure exactly how to do it, but I suspect there should be some way to configure clang to use the gnu toolchain (binutils) which is present on my Solaris system, but in /usr/gnu/bin. If anybody has an idea of how to force that at cmake/configure time I'd like to give it a try.

[nenadv]

Paul, can you try the following patch? This should create the same section under C and UPC.

diff --git a/include/llvm/MC/MCObjectFileInfo.h b/include/llvm/MC/MCObjectFileInfo.h
index 9a24cef..19599ef 100644
--- a/include/llvm/MC/MCObjectFileInfo.h
+++ b/include/llvm/MC/MCObjectFileInfo.h
@@ -83,6 +83,9 @@ protected:
   /// UPCSection - UPC shared section
   const MCSection *UPCSection;

+  /// UPCPgmSection - UPC program info section
+  const MCSection *UPCPgmSection;
+
   /// LSDASection - If exception handling is supported by the target, this is
   /// the section the Language Specific Data Area information is emitted to.
   const MCSection *LSDASection;
diff --git a/lib/MC/MCObjectFileInfo.cpp b/lib/MC/MCObjectFileInfo.cpp
index f55157d3b..b767ff7 100644
--- a/lib/MC/MCObjectFileInfo.cpp
+++ b/lib/MC/MCObjectFileInfo.cpp
@@ -425,6 +425,10 @@ void MCObjectFileInfo::InitELFMCObjectFileInfo(Triple T) {
     Ctx->getELFSection("upc_shared", ELF::SHT_NOBITS,
                        ELF::SHF_WRITE |ELF::SHF_ALLOC,
                        SectionKind::getReadOnly());
+  UPCPgmSection =
+    Ctx->getELFSection("upc_pgm_info", ELF::SHT_PROGBITS,
+                       ELF::SHF_ALLOC,
+                       SectionKind::getDataRel());
 #endif

   // Exception Handling Sections.

[nenadv]

Can you change the path? I think clang searches for the linker.

There is also |-DCMAKE_LINKER option. But it might be related to the build itself.

BTW: note that since solaris is not using the gnu linker, my build disabled the linker script. Perhaps that is related to the multiple upc_pgm_info sections?

I am not sure exactly how to do it, but I suspect there should be some way to configure clang to use the gnu toolchain (binutils) which /is/ present on my Solaris system, but in /usr/gnu/bin. If anybody has an idea of how to force that at cmake/configure time I'd like to give it a try.

— Reply to this email directly or view it on GitHub https://github.com/Intrepid/clang-upc/issues/58#issuecomment-47160276.

[PHHargrove]

I am now trying the patch from 2 comments back and will report back soon.

As for changing the path: I originally had /usr/gnu/bin in my path ahead of /usr/bin, but the result was that clang couldn't link anything because it was invoking the gnu linker with options intended for the Solaris linker. I think that not using the full path to the linker is a serious weakness in how Clang works (or fails to) on Solaris, but is a bit beyond the things I would be comfortable changing myself.

[PHHargrove]

Nenad,

Your patch for section handling appears to have done the trick for my Solaris-11/x86-64 system.

-Paul

[PHHargrove]

I see the patch was committed/pushed a I will see about a full harness run tonight.

[PHHargrove]

I didn't realize until just recently that on Solaris for x86-64 the default compiler output is 32-bit. So, I've been unknowingly testing 32-bit so far. The good news is that a full harness run produced no NEW failures other than Solaris linker issues which could occur with any compiler.

I am waiting on a 64-bit build now.

[PHHargrove]

The 64-bit build has passed the Intrepid test suite "smoke test". A full harness run is queued.

[PHHargrove]

Clang-upc with its smp runtime has now passed the full Berkeley test suite on Solaris-11/x86-64 hardware for both 32- and 64-bit ABIs.

Other than the need to manually disable the linker script (or else the build fails), this port looks complete to me. Since the linker script disable is covered as issue #60, I am closing this issue.