search

tee3 / boost-build-nvcc

A Boost.Build module for building using the NVIDIA C/C++ Compiler.
Boost Software License 1.0
0 stars 0 forks source link

Unable to use b2 with nvcc.jam #6

Open eldiener opened 5 years ago

eldiener commented 5 years ago

I am working on Windows with the latest files from the modular Boost source on github, located at E:/Programming/VersionControl/modular-boost. I put the nvcc.jam file in the Boost Build src/tools directory at E:/Programming/VersionControl/modular-boost/tools/build/src/tools. I created a user-config.jam whose contents are:

using nvcc : 9.2 : C:/Utilities/CUDA/Dev/bin/nvcc ;

where my installation of CUDA 9.2 is in the directory C:/Utilities/CUDA. I then executed the line:

b2 address-model=64 toolset=nvcc

from the preprocessor test directory at E:\Programming\VersionControl\modular-boost\libs\preprocessor\test ( I am the preprocessor maintainer ).

The result from Boost Build is:

E:/Programming/VersionControl/modular-boost/tools/build/src/build\property.jam:808: in class@property-map.find from module object(property-map)@36
error: Ambiguous key 
E:/Programming/VersionControl/modular-boost/tools/build/src/build\type.jam:334: in generated-target-ps from module type
E:/Programming/VersionControl/modular-boost/tools/build/src/build\type.jam:271: in type.generated-target-suffix from module type
E:/Programming/VersionControl/modular-boost/tools/build/src/build\virtual-target.jam:530: in virtual-target.add-prefix-and-suffix from module virtual-target
E:/Programming/VersionControl/modular-boost/tools/build/src/build\virtual-target.jam:489: in _adjust-name from module object(file-target)@5042
E:/Programming/VersionControl/modular-boost/tools/build/src/build\virtual-target.jam:274: in abstract-file-target.__init__ from module object(file-target)@5042
E:/Programming/VersionControl/modular-boost/tools/build/src/build\virtual-target.jam:582: in class@file-target.__init__ from module object(file-target)@5042
E:/Programming/VersionControl/modular-boost/tools/build/src/kernel\class.jam:90: in class.new from module class
E:/Programming/VersionControl/modular-boost/tools/build/src/build\generators.jam:576: in generated-targets from module object(C-compiling-generator)@124
E:/Programming/VersionControl/modular-boost/tools/build/src/build\generators.jam:468: in construct-result from module object(C-compiling-generator)@124
E:/Programming/VersionControl/modular-boost/tools/build/src/build\generators.jam:418: in run-really from module object(C-compiling-generator)@124
E:/Programming/VersionControl/modular-boost/tools/build/src/build\generators.jam:392: in class@generator.run from module object(C-compiling-generator)@124
E:/Programming/VersionControl/modular-boost/tools/build/src/build\generators.jam:1019: in try-one-generator-really from module generators
E:/Programming/VersionControl/modular-boost/tools/build/src/build\generators.jam:1081: in try-one-generator from module generators
E:/Programming/VersionControl/modular-boost/tools/build/src/build\generators.jam:1319: in construct-really from module generators
E:/Programming/VersionControl/modular-boost/tools/build/src/build\generators.jam:1405: in generators.construct from module generators
E:/Programming/VersionControl/modular-boost/tools/build/src/build\targets.jam:1614: in construct from module object(typed-target)@2457
E:/Programming/VersionControl/modular-boost/tools/build/src/build\targets.jam:1363: in class@basic-target.generate from module object(typed-target)@2457
E:/Programming/VersionControl/modular-boost/tools/build/src/build\targets.jam:812: in generate-really from module object(main-target)@5025
E:/Programming/VersionControl/modular-boost/tools/build/src/build\targets.jam:784: in class@main-target.generate from module object(main-target)@5025
E:/Programming/VersionControl/modular-boost/tools/build/src/build\targets.jam:933: in targets.generate-from-reference from module targets
E:/Programming/VersionControl/modular-boost/tools/build/src/build\configure.jam:365: in find-builds-raw from module configure
E:/Programming/VersionControl/modular-boost/tools/build/src/build\configure.jam:455: in configure.find-builds from module configure
E:/Programming/VersionControl/modular-boost\boostcpp.jam:734: in boostcpp.deduce-address-model from module boostcpp
E:/Programming/VersionControl/modular-boost/tools/build/src/kernel\modules.jam:107: in modules.call-in from module Jamfile<E:\Programming\VersionControl\modular-boost>
E:/Programming/VersionControl/modular-boost/tools/build/src/util\indirect.jam:105: in indirect.call from module indirect
E:/Programming/VersionControl/modular-boost/tools/build/src/build\property.jam:144: in property.evaluate-conditionals-in-context from module property
E:/Programming/VersionControl/modular-boost/tools/build/src/build\targets.jam:1087: in evaluate-requirements from module targets
E:/Programming/VersionControl/modular-boost/tools/build/src/build\targets.jam:1121: in common-properties2 from module targets
E:/Programming/VersionControl/modular-boost/tools/build/src/build\targets.jam:1017: in targets.common-properties from module targets
E:/Programming/VersionControl/modular-boost/tools/build/src/build\targets.jam:1313: in class@basic-target.generate from module object(alias-target-class)@4680
E:/Programming/VersionControl/modular-boost/tools/build/src/build\targets.jam:812: in generate-really from module object(main-target)@4942
E:/Programming/VersionControl/modular-boost/tools/build/src/build\targets.jam:784: in class@main-target.generate from module object(main-target)@4942
E:/Programming/VersionControl/modular-boost/tools/build/src/build\targets.jam:273: in class@project-target.generate from module object(project-target)@4579
E:/Programming/VersionControl/modular-boost/tools/build/src\build-system.jam:797: in load from module build-system
E:\Programming\VersionControl\modular-boost\tools\build\src/kernel\modules.jam:295: in import from module modules
E:\Programming\VersionControl\modular-boost\tools\build\src/kernel/bootstrap.jam:139: in boost-build from module
E:\Programming\VersionControl\modular-boost\boost-build.jam:17: in module scope from module

Your nvcc.jam is not working for me. Do you have any idea what is wrong and what I have tro do to get your nvcc.jam to work ? I was hoping to use your jam file in order to test Boost preprocessor code as I am dealing with a problem related to CUDA and nvcc.

tee3 commented 5 years ago

It looks to me like you have set up nvcc and used it in a reasonable way, though you have added nvcc into the main Boost.Build source, which is a little different from how I use it. I'll have to do some tests to see if I can reproduce the problem you are seeing using your setup. I think I can reproduce it locally.

Note that I have not tested nvcc as fully on WIndows as I would have liked, so there could be an issue related to that.

eldiener commented 5 years ago

How would I use nvcc.jam without adding it to the Boost Build source ? I thought that adding it to the Boost Build source makes it easiest to just use Boost Build normally, just like I would use any other toolset when invoking b2.

If you need me to test any further changes, to get it working on Windows, I will be glad to do so. It would be great to be able to use nvcc.jam on Windows since that is where the preprocessor problem which has been reported exists. I hate to have to manually manufacture an 'nvcc' command line when all the other compilers I test on Windows against Boost libraries use Boost Build and b2.

tee3 commented 5 years ago

It's actually fine and definitely the easiest way. I just don't like to modify a "system directory" of Boost.Build like that. I usually add the directory where nvcc resides to the Boost.Build path. It's definitely more complicated though. I use something like the following (I'm sure there are better ways).

# @todo HACK - add tools to the Boost.Build path

import modules ;

path-constant boost-build-nvcc : /path/to/boost-build-nvcc ;
local boost-build-path = [ modules.peek : BOOST_BUILD_PATH ] ;

modules.poke : BOOST_BUILD_PATH : $(boost-build-nvcc) $(boost-build-path) ;

I hope I can help you do what you want. It might take me a day or two to get to this, but I will get to it soon.

tee3 commented 5 years ago

@eldiener I tried this under macOS (with modular boost 1.68.0) and did not see the problems you see. I'll try the same on Windows now as well, but it's a bit more challenging as it's not my main development platform.

Basically:

Here is the procedure I used on macOS to ensure that I understand what is used for each piece.

# create a safe place to work
mkdir /tmp/nvcc-testing
cd /tmp/nvcc-testing
# create the simplest user-config.jam
echo "using nvcc ;" > ./user-config.jam
# clone modular boost
git clone --recursive --branch boost-1.68.0 https://github.com/boostorg/boost.git modular-boost
cd modular-boost
# copy the nvcc toolset to Boost.Build
cp ~/Development/tee3/boost-build-nvcc/nvcc.jam tools/build/src/tools
# Build Boost.Build
./bootstrap.sh
# build the tests for the preprocessor library
# note that this uses the local Boost.Build and only the user-config.jam above to configure it
cd libs/preprocessor/test
BOOST_BUILD_PATH=/tmp/nvcc-testing/modular-boost/tools/build /tmp/nvcc-testing/modular-boost/b2 --config=/tmp/nvcc-testing/user-config.jam address-model=64 toolset=nvcc

QUESTIONS:

tee3 commented 5 years ago

I did a similar experiment for Windows and was able to reproduce the error you saw. I will see if I can figure out the bug. Thanks!

rem create a safe place to work
mkdir C:\TEMP\nvcc-testing
cd C:\TEMP\nvcc-testing

rem create the simplest user-config.jam
echo using nvcc ; > .\user-config.jam

rem clone boost-build-nvcc
git clone https://github.com/tee3/boost-build-nvcc.git

rem clone modular boost
git clone --recursive --branch boost-1.68.0 https://github.com/boostorg/boost.git modular-boost

rem do work in modular-boost
cd modular-boost

rem copy the nvcc toolset to Boost.Build
copy C:\TEMP\nvcc-testing\boost-build-nvcc\nvcc.jam tools\build\src\tools

rem Build Boost.Build
call bootstrap.bat

rem build the tests for the preprocessor library
rem
rem note that this uses the local Boost.Build and only the
rem user-config.jam above to configure it
cd libs\preprocessor\test
set BOOST_BUILD_PATH=C:\TEMP\nvcc-testing\modular-boost\tools\build
C:\TEMP\nvcc-testing\modular-boost\b2 --config=C:\TEMP\nvcc-testing\user-config.jam address-model=64 toolset=nvcc
tee3 commented 5 years ago

The problem occurs in deduce-address-module in boostcpp.jam. This is part of the Boost release (not Boost.Build) and I don't know how it interacts with toolsets yet. I'll work on that next.

tee3 commented 5 years ago

It looks like I'm missing <address-model> in nvcc.jam. I'll figure out how that fits in and add it in to nvcc.jam.

eldiener commented 5 years ago

To answer your questions I am using the latest 'develop' from Boost source. I have no site-config.jam. The Boost Build I am using is whatever is in the latest 'develop'.

eldiener commented 5 years ago

Have you found time to look at this again in order to fix it ?

tee3 commented 5 years ago

I have, but there are a lot of issues to resolve. It works fine on macOS, but not on Windows. I believe this has to do with the way Boost.Build deals with address-model on Windows, but not on macOS. However, In order to properly deal with everything that nvcc supports (mainly passing parameters to the underlying compiler) we probably have to build nvcc on top of an underlying toolset (clang, gcc, nvcc). I'm really not sure how to proceed as Boost.Build doesn't make this easy.