Open bradcray opened 2 years ago
Though I saw this on Mojave, @aconsroe-hpe reproduced one of these failure modes on Big Sur when building from source, so I think it's independent of OS version and only based on building from source.
I believe this is from #18840 and in PR #18880 I am working on solving it. But,
However, building from Chapel from a source release or the
main
branch does not have this same problem even if those packages are installed.
If that's the case, perhaps it is a different issue from #18840 / #18880. It would be interesting to check a Homebrew build with the formula after that PR.
Anyway, note that the Homebrew formula patched main
to use brew --prefix
to add a -I
path based on it. We were previously adding /usr/local/include
to the search path but it's possible that is not happening on your system for one reason or other. AFAIK we should be doing this with brew --prefix
and getting /usr/local/include
in most Homebrew settings but that would cause the described problem if it is enabled. Until #18840 / #18880.
Thanks for the notes. I'm not sure whether this strengthens or weakens your suspicions (or has no bearing on them), but I believe that I was seeing this when building using the 1.25.0 formula from source as well. I don't think I had tried building from source prior to that formula.
[my guess: no bearing since the path ordering issue has been since time immemorial?]
I'm seeing the hwloc error today on Linux,
...
checking hwloc.h usability... yes
checking hwloc.h presence... yes
checking for hwloc.h... yes
checking for library containing chpl_hwloc_topology_init... no
configure: error: Specified topology library (hwloc_via_chapel) does not work.
make[4]: *** [Makefile:187: qthread-config] Error 1
make[3]: *** [Makefile:174: /home/fortytwo/src/chapel/third-party/qthread/install/linux64-x86_64-native-llvm-none-flat-jemalloc-bundled] Error 2
This persists after a make clean
.
DISTRIB_ID=Pop
DISTRIB_RELEASE=22.04
DISTRIB_CODENAME=jammy
DISTRIB_DESCRIPTION="Pop!_OS 22.04 LTS"
This is on the current main commit 59543df77
CHPL_TARGET_PLATFORM: linux64
CHPL_TARGET_COMPILER: llvm
CHPL_TARGET_ARCH: x86_64
CHPL_TARGET_CPU: native *
CHPL_LOCALE_MODEL: flat
CHPL_COMM: gasnet *
CHPL_COMM_SUBSTRATE: udp
CHPL_GASNET_SEGMENT: everything
CHPL_TASKS: qthreads
CHPL_LAUNCHER: amudprun
CHPL_TIMERS: generic
CHPL_UNWIND: none
CHPL_MEM: jemalloc
CHPL_ATOMICS: cstdlib
CHPL_NETWORK_ATOMICS: none
CHPL_GMP: none
CHPL_HWLOC: bundled
CHPL_RE2: bundled
CHPL_LLVM: system
CHPL_AUX_FILESYS: none
Huh... Paul, is this on a system that I/we would have access to?
Not easily -- it's my laptop.
I found I could build with CHPL_LLVM=none
.
I was also able to build without that but with CHPL_LIB_PIC=pic
.
I saw 3 attempts in the qthread configure log to find that chpl_hwloc_topology_init
symbol:
clang ... conftest.c -lm >&5
/usr/bin/ld: /tmp/conftest-184d57.o: in function `main':
conftest.c:(.text+0x12): undefined reference to `chpl_hwloc_topology_init'
clang ... conftest.c -lhwloc -lm >&5
/usr/bin/ld: /home/fortytwo/src/chapel/third-party/hwloc/install/linux64-x86_64-native-llvm-none-flat/lib/libhwloc.a(topology.o): relocation R_X86_64_32S against `.bss' can not be used when making a PIE object; recompile with -fPIE
/usr/bin/ld: failed to set dynamic section sizes: bad value
and
clang ... conftest.c -lhwloc -lnuma -lm >&5
/usr/bin/ld: cannot find -lnuma: No such file or directory
(the second of which is what led me to try CHPL_LIB_PIC
.)
The installed system llvm is
Version: 1:14.0-55~exp2
@cassella - it sounds like you are on something like Ubuntu 22.04. Do you have hwloc, jemalloc, or re2 installed? Or is it an unrelated problem to this issue's original post?
@mppf Yes, it's something like ubuntu 22.04. I don't have any of those packages installed. My issue seems more likely now to be unrelated -- I'd just initially seen the same error message and latched onto that rather than opening a new issue.
Well, whatever it was, make clobber
fixed it for me.
@bradcray — any idea if the original problem described here is still present with 1.28?
any idea if the original problem described here is still present with 1.28?
I'm giving it a try...
I ended up having to kill that build because it was dragging my entire system down. Can try again on some proverbial quiet day.
We've seen this issue again on our nightly build where we've tried to have a Homebrew formula that used our bundled version of jemalloc (and have since modified things to use a system version).
What appears to be happening is that when Homebrew builds the formula it uses its own shim version of ld (see https://github.com/Homebrew/brew/blob/master/Library/Homebrew/shims/super/ld which links to https://github.com/Homebrew/brew/blob/master/Library/Homebrew/shims/super/cc), which adds in -L
paths to existing homebrew installed libraries. You can see this by setting HOMEBREW_CC_LOG_PATH
to the name of some log file, building, and looking at the resulting log.
I'm thinking ultimately, it might be better for Chapel to give its third-party deps unique names (e.g. chpl_jemalloc.a
instead of jemalloc.a
) so there's no possibility of confusion. But this may be complicated by the fact that all these third-party deps have their own build systems, output multiple libraries, etc.
Alternatively, if we always statically link this, maybe we could just add in /absolute/path/to/our/bundled/jemalloc.a
directly to the linker command rather than passing it via -l
?
testing this today, I think only the re2
problem still exists (at least that is the only one I could recreate), as we added jemalloc
and hwloc
as brew deps in 2.1
I feel like I saw something about system re2 installations in @jabraham17 's recent changes w.r.t. these settings to get homebrew working better, but can't remember what or where… Oh, maybe I'm thinking of this line from the CHANGES.md file:
* added an error message if the user attempts to set `CHPL_RE2=system`
(and maybe I asked Jade a follow-up question about what it was and why at that time?) Anyway, Jade, can you remind us what you know about re2 here?
CHPL_RE2=system
is not supported, and as of 2.1, explicitly disabled in the build system. However, I did not address the problem of homebrews linker trying to use the homebrew installed re2 and have seen on my own system that having re2 installed breaks the chapel build from source.
The reason we don't support CHPL_RE2=system
is because we have patched the bundled re2 with new features that were not accepted into the main branch of re2. So we cannot use regular re2, unless those patches were contributed back to re2. @mppf is the expert on the details here if that is relevant to this discussion
Some of the discussion in https://github.com/chapel-lang/chapel/pull/25405 could be related to failing if re2 is installed. Or, perhaps it's a problem with Homebrew's wrappers requiring us to use the installed re2.
When building Chapel from source (
brew install --build-from-source chapel
) using the homebrew formula (which is now what happens by default if you are a Mojave user since Mojave is no longer supported), the build seems to fail if any of the following homebrew packages are installed:However, building from Chapel from a source release or the
main
branch does not have this same problem even if those packages are installed. My guess is that in homebrew's build process, homebrew's paths precede Chapel's such that their versions "win" when we're looking for headers or libraries. Uninstalling the homebrew packages in question will fix the build, but other homebrew packages may depend on their existence, so this isn't an ideal solution.So the main question here is what more we could do with our homebrew formula to avoid these conflicts when building from source?
Here are some of the failure modes in case anyone searches on some of the strings in question:
hwloc:
jemalloc:
re2: