search

clangupc / clang-upc

Clang UPC Front-End
https://clangupc.github.io/
Other
16 stars 5 forks source link

non-cmake builds fail #31

Closed PHHargrove closed 10 years ago

PHHargrove commented 10 years ago

I've been using CMake exclusively since I couldn't get my first configure-based build to work. I've come back to look again and think this is a bug (as opposed to pilot error on my part).

The "make" runs for a faily long time before failing:

make[4]: Entering directory `/global/u1/h/hargrove/upc2c/bb/tools/clang/lib/Frontend'
llvm[4]: Compiling ASTConsumers.cpp for Debug+Asserts build
llvm[4]: Compiling ASTMerge.cpp for Debug+Asserts build
llvm[4]: Compiling ASTUnit.cpp for Debug+Asserts build
llvm[4]: Compiling CacheTokens.cpp for Debug+Asserts build
llvm[4]: Compiling ChainedDiagnosticConsumer.cpp for Debug+Asserts build
llvm[4]: Compiling ChainedIncludesSource.cpp for Debug+Asserts build
llvm[4]: Compiling CompilerInstance.cpp for Debug+Asserts build
llvm[4]: Compiling CompilerInvocation.cpp for Debug+Asserts build
/global/u1/h/hargrove/upc2c/src/llvm/tools/clang/lib/Frontend/CompilerInvocation.cpp: In function 'void ParseLangArgs(clang::LangOptions&, llvm::opt::ArgList&, clang::InputKind, clang::DiagnosticsEngine&)':
/global/u1/h/hargrove/upc2c/src/llvm/tools/clang/lib/Frontend/CompilerInvocation.cpp:1368:60: error: 'UPC_PTS' was not declared in this scope
/global/u1/h/hargrove/upc2c/src/llvm/tools/clang/lib/Frontend/CompilerInvocation.cpp:1370:74: error: 'UPC_PACKED_BITS' was not declared in this scope
/global/u1/h/hargrove/upc2c/src/llvm/tools/clang/lib/Frontend/CompilerInvocation.cpp:1407:76: error: 'UPC_PTS_VADDR_ORDER' was not declared in this scope
/global/u1/h/hargrove/upc2c/src/llvm/tools/clang/lib/Frontend/CompilerInvocation.cpp:1421:59: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
/bin/rm: cannot remove `/global/u1/h/hargrove/upc2c/bb/tools/clang/lib/Frontend/Debug+Asserts/CompilerInvocation.d.tmp': No such file or directory
make[4]: *** [/global/u1/h/hargrove/upc2c/bb/tools/clang/lib/Frontend/Debug+Asserts/CompilerInvocation.o] Error 1
make[4]: Leaving directory `/global/u1/h/hargrove/upc2c/bb/tools/clang/lib/Frontend'
make[3]: *** [Frontend/.makeall] Error 2
make[3]: Leaving directory `/global/u1/h/hargrove/upc2c/bb/tools/clang/lib'
make[2]: *** [all] Error 1
make[2]: Leaving directory `/global/u1/h/hargrove/upc2c/bb/tools/clang'
make[1]: *** [clang/.makeall] Error 2
make[1]: Leaving directory `/global/u1/h/hargrove/upc2c/bb/tools'
make: *** [all] Error 1
gary-funck commented 10 years ago

On 02/21/14 00:33:16, Nenad Vukicevic wrote:

Ah, I misread your message. Thought for a moment that build got bad because of our latest update. As I recall we made only UPC related changes to the CMake build files.

To date, we build Clang UPC only with cmake.

Apart from getting the compiler to build with configure and make, setting up the configure and make for the libupc SMP-based runtime will be a bigger job.

PHHargrove commented 10 years ago

OK, I can live with that, but prior to release (delivery) I think that it should be fixed or very clearly documented that cmake is required.

gary-funck commented 10 years ago

On 02/21/14 10:25:18, Paul H. Hargrove wrote:

OK, I can live with that, but prior to release (delivery) I think that it should be fixed or very clearly documented that cmake is required.

It is useful to have this issue to track the lack of autoconf/make support. It is possible that this might be prioritized higher at some future date.

In any event, documenting the current discrepancy does seem prudent.

PHHargrove commented 10 years ago

Do we have an idea of how much work would be required to get configure-based (non-cmake) builds working? There are now multiple platforms where I've initially given up on trying clang-upc because cmake is missing or too old. At least on Solaris-11/x86-64 the up-to-date cmake doesn't build w/o changes (that were a hack unsuited to send upstream).

-Paul

gary-funck commented 10 years ago

On 03/05/14 17:41:49, Paul H. Hargrove wrote:

Do we have an idea of how much work would be required to get configure-based (non-cmake) builds working? There are now multiple platforms where I've initially given up on trying clang-upc because cmake is missing or too old. At least on Solaris-11/x86-64 the up-to-date cmake doesn't build w/o changes (that were a hack unsuited to send upstream).

No time estimate at the moment for configure and make.

An intermediate approach might be to add a configure/make that checks for cmake and if it has the correct version, runs cmake. Then, 'make' should work. Otherwise, it could add a Makefile that just touches the *.a files and builds/installs necessary header files as needed.

This might be sufficient to run clang-upc2c as the translator and upcr as the runtime?

Or ... the makefile could first download cmake source and build cmake in place ... and then run cmake ...

PHHargrove commented 10 years ago

An intermediate approach might be to add a configure/make that checks for cmake and if it has the correct version, runs cmake.

That might work, but only addresses the user's expectation that if a configure script exists at the top level then it is supposed to work. It does not eliminate the need for the external cmake tool. It would be simpler to just remove the configure script instead of making it a wrapper around cmake. It would solve exactly the same user-expects-existing-configure-script-to-work problem.

Or ... the makefile could first download cmake source and build cmake in place ... and then run cmake ...

You forgot "and patch cmake on Solaris" ;-)

nenadv commented 10 years ago

It is probably possible to build only clang-upc/clang-upc2c without the library. You can do the following:

--- a/autoconf/configure.ac
+++ b/autoconf/configure.ac
@@ -1051,6 +1051,26 @@ AC_ARG_WITH(c-include-dirs,
 AC_DEFINE_UNQUOTED(C_INCLUDE_DIRS,"$withval",
                    [Directories clang will search for headers])

+# UPC
+AC_ARG_WITH(upc-pts,
+  AS_HELP_STRING([--with-upc-pts],
+    [UPC packed or struct pointer representation.]),,
+    withval="")
+AC_DEFINE_UNQUOTED(UPC_PTS,"$withval",
+                   [UPC packed/struct pointer representation.])
+AC_ARG_WITH(upc-packed-bits,
+  AS_HELP_STRING([--with-upc-packed-bits],
+    [UPC packed pointer representation.]),,
+    withval="")
+AC_DEFINE_UNQUOTED(UPC_PACKED_PTS,"$withval",
+                   [UPC packed pointer representation.])
+AC_ARG_WITH(upc-pts-vaddr-order,
+  AS_HELP_STRING([--with-upc-pts-vaddr-order],
+    [UPC vaddr first or last.]),,
+    withval="")
+AC_DEFINE_UNQUOTED(UPC_PTS_VADDR_ORDER,"$withval",
+                   [UPC vaddr first or last.])
+

The above will make sure that file llvm/tools/clang/include/clang/Config/config.h.in gets the right values for few UPC defines required by the front-end. I see that I don't have default values above.

I'll try more on this in the morning.

nenadv commented 10 years ago

With this commit I added a Makefile for upc2c tool that is necessary for configure/make to work.

https://github.com/Intrepid/upc2c/commit/32771391b89df6a011a52f617a26d5b9a2c8b0ab

And this commit will allow for make to recurse into upc2c directory.

https://github.com/Intrepid/clang-upc/commit/e48b7c99f28ee529bfd8bb4bf62bd5598149d6f9

nenadv commented 10 years ago

Also, added -fupc-pre-include option to clang-upc. It might be useful to invoke the clang-upc without having to provide a header file (when libupc is missing). For upc2c translator we can always enforce this option, thus removing the need for empty clang-upc.h and clang-upc-lib.h in UPCR structure.

https://github.com/Intrepid/clang-upc/commit/ba78885d011d5e46d52739ff83a6f83864a74f79

nenadv commented 10 years ago

With this commit to LLVM_UPC https://github.com/Intrepid/llvm-upc/commit/91d4f58f02293c69debb5142f5fe0be020305f74 , plus couple of others mentioned on this thread, we should be able to build clang-upc and clang-upc2c with configure/Makefile structure without building the libupc. In essence, this is just a partial fix.

After looking at the LLVM configure it seems that they expect all the configure to happen in one place (autoconf/configure.ac) for all the components (in our case llvm, clang, and clang tools). At the moment there is no link for doing libupc configure/make through this kind of build. I have to see some other clang tools and figure out what would make "make" descent into the libupc and configure/make.

PHHargrove commented 10 years ago

Nenad,

Thanks. I will try on x86-64 and it that works as expected I'll give x86-32 a try.

-Paul

PHHargrove commented 10 years ago

Nenad,

One oddity I see is an un-expanded version number:

$ ./bin/clang --version
clang version 3.4  (UPC @CLANG_UPC_VERSION@)
Target: i386-unknown-freebsd9.1
Thread model: posix

This is breaking the driver's regex for version reporting. However, the good news is in the Target: line of that output.

nenadv commented 10 years ago

Fixed the version string with https://github.com/Intrepid/clang-upc/commit/0218bdcab0d4490cd4de8e9879d545c1a0931a6a

I was surprised to see that my version string contained repository name strings. And it looks like it happens only if we use configure/Makefile build option.

PHHargrove commented 10 years ago

Nenad,

I can now get a build of clang-upc via configure (instead of cmake). However, the resulting build doesn't work because of something bogus in the generated linker script:

{hargrove@cvrsvc01 llvm-upc}$ ./bin/clang-upc ~/hello.upc 
/usr/bin/ld:/global/scratch2/sd/hargrove/llvm-upc/bin/../lib/upc.ld:248: syntax error
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Line 248 (and context) in the linker script looks like:

$ cat -n /global/scratch2/sd/hargrove/llvm-upc/bin/../lib/upc.ld| grep -wC5 248
   243    .debug_funcnames 0 : { *(.debug_funcnames) }
   244    .debug_typenames 0 : { *(.debug_typenames) }
   245    .debug_varnames  0 : { *(.debug_varnames) }
   246    /* DWARF 3 */
   247    .debug_pubtypes 0 : { *(.debug_pubtypes) }
   248    .debug_ranges   /usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../lib64/crt1.o: In function `_start':
   249  (.text+0x20): undefined reference to `main'
   250  0 : { *(.debug_ranges) }
   251    /DISCARD/ : { *(.note.GNU-stack) }
   252  }

suggesting to me that something went seriously wrong in its generation.

This is Linux/x86-64, using gcc-4.7.0 and binutils "2.17.50.0.6-14.el5 20061020"

nenadv commented 10 years ago

Hmm I'll take a look. Linux x86-64 is our main development platform and I also tested on PPC64.

Note that with the configure build you only get one flavor of the libupc (packed by default). To duplicate what cmake is capable might will take much more work.

On 6/11/14 5:25 PM, Paul H. Hargrove wrote:

Nenad,

I can now get a build of clang-upc via configure (instead of cmake). However, the resulting build doesn't work because of something bogus in the generated linker script:

{hargrove@cvrsvc01 llvm-upc}$ ./bin/clang-upc ~/hello.upc /usr/bin/ld:/global/scratch2/sd/hargrove/llvm-upc/bin/../lib/upc.ld:248: syntax error clang: error: linker command failed with exit code 1 (use -v to see invocation)

Line 248 (and context) in the linker script looks like:

$ cat -n /global/scratch2/sd/hargrove/llvm-upc/bin/../lib/upc.ld grep -wC5 248 243 .debugfuncnames 0 : { (.debug_funcnames) } 244 .debug_typenames 0 : { (.debug_typenames) } 245 .debug_varnames 0 : { (.debugvarnames) } 246 / DWARF 3 / 247 .debug_pubtypes 0 : { (.debug_pubtypes) } 248 .debug_ranges /usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../lib64/crt1.o: In function _start': 249 (.text+0x20): undefined reference tomain' 250 0 : { (.debug_ranges) } 251 /DISCARD/ : { (.note.GNU-stack) } 252 }

suggesting to me that something went seriously wrong in its generation.

This is Linux/x86-64, using gcc-4.7.0 and binutils "2.17.50.0.6-14.el5 20061020"

— Reply to this email directly or view it on GitHub https://github.com/Intrepid/clang-upc/issues/31#issuecomment-45817041.

nenadv commented 10 years ago

What configure options do you use?

PHHargrove commented 10 years ago

Nenad asks

What configure options do you use?

--prefix=[...]  --disable-jit --disable-docs --enable-keep-symbols  --enable-clang-upc2c --enable-targets=host   --enable-optimized
PHHargrove commented 10 years ago

Nenad,

This is on carver.nersc.gov where you should have an account. The --enable-optimized --enable-keep-symbols is a compromise to avoid the OOM link failure I encounter for a normal debug build. Disabling jit and docs just saves time and space.

If you do try to reproduce on Carver, you will need to module load gcc to get a GCC new enough to build llvm and clang.

-Paul

PHHargrove commented 10 years ago

The commands to build upc.ld seem to differ depending on cmake vs configure. Specifically cmake runs ld and Makefile uses $(CC) $(LDFLAGS) ...

In /runtime/libupc/CMakeLists.txt

add_custom_command(OUTPUT ${upc_link_script}
   COMMAND ld --verbose | ${PERL_EXECUTABLE} ${PROJECT_SOURCE_DIR}/gen-upc-ld-script.pl > ${upc_link_script}
   VERBATIM)

And in runtime/libupc/Makefile

$(UPC_LINK_SCRIPT): gen-upc-ld-script.pl
        $(Verb) $(CC) $(LDFLAGS) -Xlinker --verbose -x none /dev/null 2>&1 | \
        $(PERL) $+ > $@

The precise placement of the bogus "undefined reference to `main'" message seems to be irreproducable, and looks to be the result of inter mixing stderr and stdout.