Link failures (pthreads) on OpenBSD

[PHHargrove]

All clang-upc compilations on OpenBSD are now failing with:

/home/phargrov/upcnightly/llvm-upc/bin/../lib/libupc.a(upc_main.o)(.text+0xc): In function `__upc_omp_check':
: undefined reference to `pthread_self'
/home/phargrov/upcnightly/llvm-upc/bin/../lib/libupc.a(upc_main.o)(.text+0x1df6): In function `main':
: undefined reference to `pthread_self'
/home/phargrov/upcnightly/llvm-upc/bin/../lib/libupc.a(upc_main.o)(.text+0x1ed7): In function `main':
: undefined reference to `pthread_self'
clang: error: linker command failed with exit code 1 (use -v to see invocation)
gmake: *** [RandomAccess_UPC] Error 1

This clang-upc build was configured via autoconf, in case that matters.

-Paul

[PHHargrove]

I should have mentioned that the problem appeared between test runs on May 26 (with no problem) and May 31 (all tests failed to link). I didn't have any OpenBSD+clang-upc tests between those two dates.

[nenadv]

It seems that -lpthread is missing on the linker line. I am not sure why linux does not require it.

I also noticed the following linker warning on openbsd:

/home/phargrov/upcnightly/llvm-upc/bin/../lib/libupc.a(upc_main.o)(.text+0x181): In function `__upc_fatal':
: warning: vsprintf() is often misused, please use vsnprintf()
/home/phargrov/upcnightly/llvm-upc/bin/../lib/libupc.a(upc_main.o)(.text+0x18a6): In function `main':
: warning: strcpy() is almost always misused, please use strlcpy()
/home/phargrov/upcnightly/llvm-upc/bin/../lib/libupc.a(upc_main.o)(.text+0xc1): In function `__upc_fatal':
: warning: sprintf() is often misused, please use snprintf()
-bash-4.3$ /home/phargrov/upcnightly/llvm-upc/bin/clang-upc -o t t.upc -lpthread   
/home/phargrov/upcnightly/llvm-upc/bin/../lib/libupc.a(upc_main.o)(.text+0x181): In function `__upc_fatal':
: warning: vsprintf() is often misused, please use vsnprintf()
/home/phargrov/upcnightly/llvm-upc/bin/../lib/libupc.a(upc_main.o)(.text+0x18a6): In function `main':
: warning: strcpy() is almost always misused, please use strlcpy()
/home/phargrov/upcnightly/llvm-upc/bin/../lib/libupc.a(upc_main.o)(.text+0xc1): In function `__upc_fatal':
: warning: sprintf() is often misused, please use snprintf()

[PHHargrove]

Nenad,

1.

OpenBSD says to use "-pthread" in CFLAGS and LDFLAGS, and so do FreeBSD and NetBSD. You are not expected to pass "-D_REENTRANT" to CC nor "-lpthread" to LD. Brief testing shows clang accepts "-pthread" on those three BSD platforms AND on Linux and Solaris. So, I think that is the option you should be using.

OpenBSD in particular does NOT recommend "-lpthreads" because their system has gone through multiple (at least 2, IIRC) distinct implementations of POSIX threads and libpthread.{a,so} might not be the right one.

2.

The linker warnings are a "feature" on security-centric OpenBSD.

The harness normally filters these away. They only show in the nightly test output because in the event an error or non-filtered warning appears we emit the unfiltered stdout and stderr.

The warnings are generated once per symbol which means if you fix the warnings above you will just change the warning to reference the next instance of each of those functions. Starting down that path is a looong journey, complicated by the fact that the recommended "strlcpy()" family of replacements for "strncpy()" and related functions are not present on most platforms.

-Paul

[nenadv]

-pthread on the command line should be ok. But ultimately this will end up with -lpthread on the 'ld' line. Right? We already selectively add pthreads for the portals runtime and we also need it if OMP checks are enabled.

On 6/8/15 2:17 PM, Paul H. Hargrove wrote:

Nenad,

1.

OpenBSD says to use "-pthread" in CFLAGS and LDFLAGS, and so do FreeBSD and NetBSD. You are not expected to pass "-D_REENTRANT" to CC nor "-lpthread" to LD. Brief testing shows clang accepts "-pthread" on the platforms, AND on Linux and Solaris. So, I think that is the option you should be using.

OpenBSD in particular does NOT recommend "-lpthreads" because their system has gone through multiple (at least 2, IIRC) distinct implementations of POSIX threads and libpthread.{a,so} might not be the right one.

2.

The linker warnings are a "feature" on security-centric OpenBSD.

The harness normally filters these away. They only show in the nightly test output because in the event an error or non-filtered warning appears we emit the unfiltered stdout and stderr.

The warnings are generated once per symbol which means if you fix the warnings above you will just change the warning to reference the next instance of each of those functions. Starting down that path is a looong journey, complicated by the fact that the recommended "strlcpy()" family of replacements for "strncpy()" and related functions are not present on most platforms.

-Paul

— Reply to this email directly or view it on GitHub https://github.com/Intrepid/clang-upc/issues/78#issuecomment-110142926.

[PHHargrove]

Nenad asks:

-pthread on the command line should be ok. But ultimately this will end up with -lpthread on the 'ld' line. Right?

That is correct today. In the past -pthread could have linked librthread ("r" in place of "p"), and in the future it may link something else.

-Paul

[PHHargrove]

Just for completeness, here is the relevant goop from lib/Basic/Targets.cpp and lib/Driver/Tools.cpp. They translate -pthread into -D_REENTRANT and -lpthreads (or -lpthread_p), respectively.

class OpenBSDTargetInfo : public OSTargetInfo<Target> {
protected:
  void getOSDefines(const LangOptions &Opts, const llvm::Triple &Triple,
                    MacroBuilder &Builder) const override {
    // OpenBSD defines; list based off of gcc output

    Builder.defineMacro("__OpenBSD__");
    DefineStd(Builder, "unix", Opts);
    Builder.defineMacro("__ELF__");
    if (Opts.POSIXThreads)
      Builder.defineMacro("_REENTRANT");
  }
[...]

void openbsd::Link::ConstructJob(
[...]
    if (Args.hasArg(options::OPT_pthread)) {
      if (!Args.hasArg(options::OPT_shared) &&
          Args.hasArg(options::OPT_pg))
         CmdArgs.push_back("-lpthread_p");
      else
         CmdArgs.push_back("-lpthread");
    }
[...]

However, on NetBSD the preprocessor defines _POSIX_THREADS instead of _REENTRANT. That is the sort of platform-specific thing use of -pthread is meant to hide.

So, I feel that the right behavior would be for the driver to assert the "-pthread" option for (at least) preprocess and link stages when the language is UPC, rather than assuming -D_REENTRANT and -lpthread. Of course I say that without having any idea how one would achieve this.

[nenadv]

Fixed by https://github.com/Intrepid/clang-upc/commit/cbec6de5ce1503b0214632583f54bf6fce5f9357