Open cocoza4 opened 1 year ago
Hi @cocoza4
Thanks for your question.
Some quick feedback:
conanfile.py
to the minimal case, aren't you? If not, it seems the def package_info(self):
method, for example, can be missing information about libraries, etc.toolchain_msvc-x64/0.1.0
recipe contents could be very important. Who and how is injecting the toolchain_msvc-x64.cmake
toolchain?@memsharded thanks for your response, I updated conan to 1.60.2 and it still didn't work.
for msvc toolchain, I ran conan create toolchain-msvc.py
. please find the script below
class MSVCToolRequire(ConanFile):
name = "toolchain_msvc-x64"
version = "0.1.0"
exports_sources = "../../cmake/toolchains/msvc-x64.cmake"
def package(self):
path = os.path.join(self.source_folder, "msvc-x64.cmake")
shutil.copy(path, self.package_folder)
def package_info(self):
path = os.path.join(self.package_folder, "msvc-x64.cmake")
if not os.path.exists(path):
raise ConanException("Toolchain file not found!")
self.conf_info.define("tools.cmake.cmaketoolchain:user_toolchain", [path])
Thanks for sharing it!
The conan build
command in Conan 1.X might have some limitations. I guess that you have tried with conan create .
instead the full install + source + build
flow, and it fails in the same way? If you haven't, could you please try that?
Maybe sharing the full output, not only the build part, could have some extra hints. Thanks!
root@peeranat:/workspace/3rdparty/crc32c# conan create . -pr:h /workspace/conan/profiles/msvc-x64 -pr:b default
[HOOK - attribute_checker.py] pre_export(): WARN: Conanfile doesn't have 'url'. It is recommended to add it as attribute
Exporting package recipe
crc32c/1.1.2: The stored package has not changed
crc32c/1.1.2: Exported revision: 9a7e165e7628a58a72fcd16e6f5749db
Configuration (profile_host):
[settings]
arch=x86_64
build_type=Release
compiler=clang
compiler.cppstd=17
compiler.libcxx=libstdc++11
compiler.version=10
os=Windows
[options]
[build_requires]
*: toolchain_msvc-x64/0.1.0
[env]
[conf]
tools.env.virtualenv:auto_use=True
[buildenv]
CXXFLAGS="/EHsc"
Configuration (profile_build):
[settings]
arch=x86_64
arch_build=x86_64
build_type=Release
compiler=gcc
compiler.libcxx=libstdc++11
compiler.version=9
os=Linux
os_build=Linux
[options]
[build_requires]
[env]
crc32c/1.1.2: Forced build from source
Installing package: crc32c/1.1.2
Requirements
crc32c/1.1.2 from 'artifactory' - Cache
Packages
crc32c/1.1.2:0246896ed8e77a58a3985dffb2e82bf25f71ed9e - Build
Build requirements
toolchain_msvc-x64/0.1.0 from 'artifactory' - Cache
Build requirements packages
toolchain_msvc-x64/0.1.0:5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9 - Cache
Cross-build from 'Linux:x86_64' to 'Windows:x86_64'
Installing (downloading, building) binaries...
toolchain_msvc-x64/0.1.0: Already installed!
crc32c/1.1.2: Applying build-requirement: toolchain_msvc-x64/0.1.0
crc32c/1.1.2: WARN: Build folder is dirty, removing it: /root/.conan/data/crc32c/1.1.2/_/_/build/0246896ed8e77a58a3985dffb2e82bf25f71ed9e
crc32c/1.1.2: Copying sources to build folder
crc32c/1.1.2: Building your package in /root/.conan/data/crc32c/1.1.2/_/_/build/0246896ed8e77a58a3985dffb2e82bf25f71ed9e
crc32c/1.1.2: Generator txt created conanbuildinfo.txt
crc32c/1.1.2: Calling generate()
crc32c/1.1.2: Preset 'release' added to CMakePresets.json. Invoke it manually using 'cmake --preset release'
crc32c/1.1.2: If your CMake version is not compatible with CMakePresets (<3.19) call cmake like: 'cmake <path> -G "Unix Makefiles" -DCMAKE_TOOLCHAIN_FILE=/root/.conan/data/crc32c/1.1.2/_/_/build/0246896ed8e77a58a3985dffb2e82bf25f71ed9e/build/Release/generators/conan_toolchain.cmake -DCMAKE_POLICY_DEFAULT_CMP0091=NEW -DCMAKE_BUILD_TYPE=Release'
crc32c/1.1.2: Aggregating env generators
crc32c/1.1.2: Calling build()
crc32c/1.1.2: CMake command: cmake -G "Unix Makefiles" -DCMAKE_TOOLCHAIN_FILE="/root/.conan/data/crc32c/1.1.2/_/_/build/0246896ed8e77a58a3985dffb2e82bf25f71ed9e/build/Release/generators/conan_toolchain.cmake" -DCMAKE_INSTALL_PREFIX="/root/.conan/data/crc32c/1.1.2/_/_/package/0246896ed8e77a58a3985dffb2e82bf25f71ed9e" -DCMAKE_POLICY_DEFAULT_CMP0091="NEW" -DCMAKE_BUILD_TYPE="Release" "/root/.conan/data/crc32c/1.1.2/_/_/build/0246896ed8e77a58a3985dffb2e82bf25f71ed9e/./crc32c"
-- Using Conan toolchain: /root/.conan/data/crc32c/1.1.2/_/_/build/0246896ed8e77a58a3985dffb2e82bf25f71ed9e/build/Release/generators/conan_toolchain.cmake
-- Conan toolchain: C++ Standard 17 with extensions OFF
-- The C compiler identification is MSVC 19.29.30133.0
-- The CXX compiler identification is MSVC 19.29.30133.0
-- Check if the system is big endian
-- Searching 16 bit integer
-- Looking for sys/types.h
-- Looking for sys/types.h - found
-- Looking for stdint.h
-- Looking for stdint.h - found
-- Looking for stddef.h
-- Looking for stddef.h - found
-- Check size of unsigned short
-- Check size of unsigned short - done
-- Searching 16 bit integer - Using unsigned short
-- Check if the system is big endian - little endian
-- Performing Test CRC32C_HAVE_NO_DEPRECATED
-- Performing Test CRC32C_HAVE_NO_DEPRECATED - Failed
-- Performing Test CRC32C_HAVE_NO_SIGN_COMPARE
-- Performing Test CRC32C_HAVE_NO_SIGN_COMPARE - Failed
-- Performing Test CRC32C_HAVE_NO_UNUSED_PARAMETER
-- Performing Test CRC32C_HAVE_NO_UNUSED_PARAMETER - Failed
-- Performing Test CRC32C_HAVE_NO_MISSING_FIELD_INITIALIZERS
-- Performing Test CRC32C_HAVE_NO_MISSING_FIELD_INITIALIZERS - Failed
-- Performing Test HAVE_BUILTIN_PREFETCH
-- Performing Test HAVE_BUILTIN_PREFETCH - Failed
-- Performing Test HAVE_MM_PREFETCH
-- Performing Test HAVE_MM_PREFETCH - Failed
-- Performing Test HAVE_SSE42
-- Performing Test HAVE_SSE42 - Failed
-- Performing Test HAVE_ARM64_CRC32C
-- Performing Test HAVE_ARM64_CRC32C - Failed
-- Performing Test HAVE_STRONG_GETAUXVAL
-- Performing Test HAVE_STRONG_GETAUXVAL - Failed
-- Performing Test HAVE_WEAK_GETAUXVAL
-- Performing Test HAVE_WEAK_GETAUXVAL - Failed
-- Configuring done
-- Generating done
-- Build files have been written to: /root/.conan/data/crc32c/1.1.2/_/_/build/0246896ed8e77a58a3985dffb2e82bf25f71ed9e/build/Release
crc32c/1.1.2: CMake command: cmake --build "/root/.conan/data/crc32c/1.1.2/_/_/build/0246896ed8e77a58a3985dffb2e82bf25f71ed9e/build/Release" '--' '-j12'
[ 20%] Building CXX object CMakeFiles/crc32c_arm64.dir/src/crc32c_arm64.cc.obj
[ 40%] Building CXX object CMakeFiles/crc32c_sse42.dir/src/crc32c_sse42.cc.obj
c1xx: fatal error C1083: Cannot open source file: '': No such file or directory
crc32c_sse42.cc
Generating Code...
make[2]: *** [CMakeFiles/crc32c_sse42.dir/build.make:76: CMakeFiles/crc32c_sse42.dir/src/crc32c_sse42.cc.obj] Error 2
make[2]: *** Deleting file 'CMakeFiles/crc32c_sse42.dir/src/crc32c_sse42.cc.obj'
make[1]: *** [CMakeFiles/Makefile2:113: CMakeFiles/crc32c_sse42.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
c1xx: fatal error C1083: Cannot open source file: '': No such file or directory
crc32c_arm64.cc
Generating Code...
make[2]: *** [CMakeFiles/crc32c_arm64.dir/build.make:76: CMakeFiles/crc32c_arm64.dir/src/crc32c_arm64.cc.obj] Error 2
make[2]: *** Deleting file 'CMakeFiles/crc32c_arm64.dir/src/crc32c_arm64.cc.obj'
make[1]: *** [CMakeFiles/Makefile2:87: CMakeFiles/crc32c_arm64.dir/all] Error 2
make: *** [Makefile:124: all] Error 2
crc32c/1.1.2:
crc32c/1.1.2: ERROR: Package '0246896ed8e77a58a3985dffb2e82bf25f71ed9e' build failed
crc32c/1.1.2: WARN: Build folder /root/.conan/data/crc32c/1.1.2/_/_/build/0246896ed8e77a58a3985dffb2e82bf25f71ed9e/build/Release
ERROR: crc32c/1.1.2: Error in build() method, line 36
cmake.build()
ConanException: Error 2 while executing cmake --build "/root/.conan/data/crc32c/1.1.2/_/_/build/0246896ed8e77a58a3985dffb2e82bf25f71ed9e/build/Release" '--' '-j12'
root@peeranat:/workspace/3rdparty/crc32c#
Checking the output I can't see anything relevant. The next step would be to reproduce it locally, but I am afraid that setting up that cross-compiler might be not simple.
Does the cross-build of other projects work without problems with that setup? You might try with an out-of-the box conan new hello/0.1 -m=cmake_lib
recipe, to see what happens.
I think this could be the use of check_cxx_source_compiles()
and other similar CMake utilities. For some reason sometimes they don't play well when cross-building with Conan, but we need to further investigate it.
I confirm that the cross compiling works on other project. And you are right, check_cxx_source_compiles()
caused the problem, this command should set HAVE_MM_PREFETCH
and HAVE_SSE42
to ON
instead of OFF
.
I have tried to make this fail with some simpler cross-build scenarios (Linux86->LinuxArm and Linux86->WindowsMingw64), but without success, your recipe above with something like this cross-builds fine:
[settings]
os=Windows
arch=x86_64
compiler=gcc
compiler.libcxx=libstdc++11
compiler.cppstd=14
compiler.version=9
build_type=Release
[conf]
tools.build:compiler_executables = {"c": "x86_64-w64-mingw32-gcc", "cpp": "x86_64-w64-mingw32-g++"}
can you try with MSVC?
seems like the problem only occurs with MSVC
Setting up MSVC in Linux seems way more complicated and that it will require more time. Is there any docker image or something like that with things pre-installed we could use to try thing?
@memsharded MSVC on linux can be found here: https://drive.google.com/file/d/1PPALM6jzUMQQ-5IofAGsL5-WOH1-oifn/view?usp=sharing
@memsharded is everything ok, anything else you need from my side?
I am afraid that I might not be allowed to use such a binary in my computer, this is why I was asking for some docker image, I was expecting that the installation of MSVC on linux could be done from some public official sources, and the dockerfile would help in setting it up too. I will try to check it, but I might not be able, I will keep you posted.
i managed to find the root cause, when I commented the following code, the build passed. Any idea why's that?
[conf]
tools.env.virtualenv:auto_use=True
[buildenv]
CXXFLAGS="/EHsc"
That code is creating conanbuild.sh
or conanbuild.bat
that are loaded before executing commands, and they contain information about the environment, like things in tool_requires
and such.
I don't know exactly why they would be affecting. Does it work if you only comment the CXXFLAGS
but leave the [conf]
section enabled?
yes it worked when I commented out CXXFLAGS
. Is there a way I can remove CXXFLAGS
in conanfile.py
?
@memsharded any update on this?
What is your question?
Hi,
I'm unable to cross-compile https://github.com/google/crc32c using MSVC. However, normal build without Conan works
Steps
conan source .
conan install . -pr:h /workspace/conan/profiles/msvc-x64 -pr:b default
conan build .
Output
The problem comes from
HAVE_MM_PREFETCH - Failed
andHAVE_SSE42 - Failed
. When building without Conan, they were bothSuccess
.Conan file
msvc-x64
toolchain
toolchain_msvc-x64
Note that this toolchain works when building other libraries.
Conan 1.5.8
Have you read the CONTRIBUTING guide?