search

bfgroup / b2

B2 makes it easy to build C++ projects, everywhere.
https://www.bfgroup.xyz/b2/
Boost Software License 1.0
76 stars 228 forks source link

clang executable detection #27

Closed sdarwin closed 3 years ago

sdarwin commented 3 years ago

Environment and version details

Describe your use case

Hi,

Here's an issue that I've just encounter.

Imagine you run a build with "toolset=clang-10"

For example, 

./b2 libs/json/test toolset=clang-10 cxxstd=17,2a variant=release -j3 warnings=extra warnings-as-errors=on

What is the expected result if clang-10 doesn't exist on that machine? You would expect a warning message would appear. Even more likely, that the build would stop and tell you the compiler is not available.

However, what b2 does is proceed with whatever clang++ that it finds on the system.

By extension, if multiple clang's are installed, which one will it use? Not what is specified by toolset=. Rather, the first clang++ it finds on the PATH.

jeking3 already discovered this, mentioned here: https://github.com/boostorg/build/issues/306 .

Describe the solution you'd like

gcc does not have the problem. "it would seem that the version selection code currently inside the init rule inside gcc.jam should be considered generic to all toolset= processing."
Make the gcc solution more general, and include clang.

Kojoley commented 3 years ago
  • B2 Version:
+ b2 -v
B2 Version 4.5. OS=LINUX.

You had tested not the latest version. The #1 should have solved exactly the issue you are describing.

sdarwin commented 3 years ago

@Kojoley , if that's the case, then excellent news!

Next question.. If you build a boost library (with boost and relevant dependencies) on the 'develop' branch instead of the 'master' branch, it usually pulls in the latest 'develop' branch code from dependent libraries. However, the most recent copy of b2 was a month ago:

commit 2ad7c03b1e5669842b6e5106632cd6cff8cffcb8
Author: Automated Commit <automated@calamity.org.uk>
Date:   Tue Apr 6 13:00:03 2021 +0000

    Update build from develop

Can the new version of b2 be commited into boost 'develop', so that it will apply to CI testing?

Kojoley commented 3 years ago

I do not understand why development had moved here and what is the synchronization policy. These questions must be addressed to @grafikrobot.

sdarwin commented 3 years ago

@Kojoley thank you for the feature enhancement! The easiest way to test and verify will be in automatic CI, so hopefully it can be merged into boost, or the standard boost scripts. Closing the issue.

grafikrobot commented 3 years ago

I do not understand why development had moved here and what is the synchronization policy. These questions must be addressed to @grafikrobot.

See.. https://github.com/bfgroup/b2/discussions/29

sdarwin commented 3 years ago

Hi,

The new version of clang detection was merged into boostorg/build a few days ago. Running tests, and something unusual seems to be happening. clang-9, clang-10, clang-11 are succeeding. clang 3.8, 4.0, 5.0, 6.0, 7, 8 are failing.

++ ./b2 libs/json/test toolset=clang-8 cxxstd=17 -j3 warnings=extra warnings-as-errors=on
/home/runner/work/json/boost-root/tools/build/src/tools/common.jam:1140: in common.find-compiler from module common
error: toolset clang-linux initialization:
error: version '8' requested but 'clang++-8' not found and version '4.2.1' of default 'clang++-8' does not match

To debug, I added a new section "DEBUG 2", as follows:

echo "PATH IS $PATH" || true
ls -al /usr/bin/clang* || true

It shows that clang++-8 exists and is in the PATH.

example jobs: https://github.com/samd2/json/actions/runs/919878566

Did you already test those combinations? (such as clang 5 ?)

Kojoley commented 3 years ago

That's strange, could you please add --debug-configuration to b2 invocation and post the log?

sdarwin commented 3 years ago

Here is --debug-configuration

++ ./b2 libs/json/test toolset=clang-8 cxxstd=17 -j3 warnings=extra warnings-as-errors=on --debug-configuration
notice: found boost-build.jam at /home/runner/work/json/boost-root/boost-build.jam
notice: loading B2 from /home/runner/work/json/boost-root/tools/build/src/bootstrap.jam
notice: Searching '/etc' '/home/runner' '/home/runner/work/json/boost-root/tools/build/src' '/home/runner/work/json/boost-root/tools/build/src/kernel' '/home/runner/work/json/boost-root/tools/build/src/util' '/home/runner/work/json/boost-root/tools/build/src/build' '/home/runner/work/json/boost-root/tools/build/src/tools' '/home/runner/work/json/boost-root/tools/build/src/contrib' '/home/runner/work/json/boost-root/tools/build/src/.' for site-config configuration file 'site-config.jam'.
notice: Configuration file 'site-config.jam' not found in '/etc' '/home/runner' '/home/runner/work/json/boost-root/tools/build/src' '/home/runner/work/json/boost-root/tools/build/src/kernel' '/home/runner/work/json/boost-root/tools/build/src/util' '/home/runner/work/json/boost-root/tools/build/src/build' '/home/runner/work/json/boost-root/tools/build/src/tools' '/home/runner/work/json/boost-root/tools/build/src/contrib' '/home/runner/work/json/boost-root/tools/build/src/.'.
notice: Searching '/home/runner' '/home/runner/work/json/boost-root/tools/build/src' '/home/runner/work/json/boost-root/tools/build/src/kernel' '/home/runner/work/json/boost-root/tools/build/src/util' '/home/runner/work/json/boost-root/tools/build/src/build' '/home/runner/work/json/boost-root/tools/build/src/tools' '/home/runner/work/json/boost-root/tools/build/src/contrib' '/home/runner/work/json/boost-root/tools/build/src/.' for user-config configuration file 'user-config.jam'.
notice: Configuration file 'user-config.jam' not found in '/home/runner' '/home/runner/work/json/boost-root/tools/build/src' '/home/runner/work/json/boost-root/tools/build/src/kernel' '/home/runner/work/json/boost-root/tools/build/src/util' '/home/runner/work/json/boost-root/tools/build/src/build' '/home/runner/work/json/boost-root/tools/build/src/tools' '/home/runner/work/json/boost-root/tools/build/src/contrib' '/home/runner/work/json/boost-root/tools/build/src/.'.
notice: Searching '.' for project-config configuration file 'project-config.jam'.
notice: Loading project-config configuration file 'project-config.jam' from '.'.
notice: will use 'clang++' for clang-linux, condition <toolset>clang-linux-4
notice: [python-cfg] Configuring python...
notice: [python-cfg]   user-specified version: "2.7"
notice: [python-cfg]   user-specified cmd-or-prefix: "/usr"
notice: [python-cfg] Checking interpreter command "/usr/bin/python2.7"...
notice: [python-cfg] running command '/usr/bin/python2.7 -c "from sys import *; print('version=%d.%d\nplatform=%s\nprefix=%s\nexec_prefix=%s\nexecutable=%s' % (version_info[0],version_info[1],platform,prefix,exec_prefix,executable))" 2>&1'
notice: [python-cfg] ...requested configuration matched!
notice: [python-cfg] Details of this Python configuration:
notice: [python-cfg]   interpreter command: "/usr/bin/python2.7"
notice: [python-cfg]   include path: "/usr/include/python2.7"
notice: [python-cfg]   library path: "/usr/lib/python2.7/config" "/usr/lib"
notice: [python-cfg] Checking for NumPy...
notice: [python-cfg] running command '/usr/bin/python2.7 -c "import sys; sys.stderr = sys.stdout; import numpy; print(numpy.get_include())"'
notice: [python-cfg] NumPy disabled. Reason:
notice: [python-cfg]   /usr/bin/python2.7 -c "import sys; sys.stderr = sys.stdout; import numpy; print(numpy.get_include())" aborted with 
notice: [python-cfg]   Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: No module named numpy
notice: [cmdline-cfg] toolset clang-8 not previously configured; attempting to auto-configure now
/home/runner/work/json/boost-root/tools/build/src/tools/common.jam:1140: in common.find-compiler from module common
error: toolset clang-linux initialization:
error: version '8' requested but 'clang++-8' not found and version '4.2.1' of default 'clang++-8' does not match
error: initialized from
/home/runner/work/json/boost-root/tools/build/src/tools/clang-linux.jam:55: in clang-linux.init from module clang-linux
/home/runner/work/json/boost-root/tools/build/src/build/toolset.jam:44: in toolset.using from module toolset
/home/runner/work/json/boost-root/tools/build/src/tools/clang.jam:33: in clang.init from module clang
/home/runner/work/json/boost-root/tools/build/src/build/toolset.jam:44: in toolset.using from module toolset
/home/runner/work/json/boost-root/tools/build/src/build-system.jam:543: in process-explicit-toolset-requests from module build-system
/home/runner/work/json/boost-root/tools/build/src/build-system.jam:610: in load from module build-system
/home/runner/work/json/boost-root/tools/build/src/kernel/modules.jam:295: in import from module modules
/home/runner/work/json/boost-root/tools/build/src/kernel/bootstrap.jam:139: in boost-build from module
/home/runner/work/json/boost-root/boost-build.jam:17: in module scope from module

Error: Process completed with exit code 1.
Kojoley commented 3 years ago

I get it, Clang 8 and below reports its version as 4.2.1

sdarwin commented 3 years ago

Well, you may be right. Certainly, something is reporting 4.2.1.
Querying the version on the command line shows 8.0.1. 

clang++-8 --version
clang version 8.0.1- (branches/release_80)
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
pdimov commented 3 years ago

Yes, clang 8 reports the GCC emulation version on clang++ -dumpversion (and does not support -dumpfullversion).

C:\Users\Peter Dimov>clang++ --version
clang version 8.0.1 (tags/RELEASE_801/final)
Target: x86_64-unknown-windows-cygnus
Thread model: posix
InstalledDir: /usr/bin

C:\Users\Peter Dimov>clang++ -dumpversion
4.2.1

C:\Users\Peter Dimov>clang++ -dumpfullversion
clang-8: error: no input files

C:\Users\Peter Dimov>clang++ -dumpfullversion -dumpversion
4.2.1