conan-io / conan

Conan - The open-source C and C++ package manager
https://conan.io
MIT License
8.27k stars 981 forks source link

[question] Unable to disable bitcode for visionOS #16988

Closed shiena closed 2 months ago

shiena commented 2 months ago

What is your question?

I am building joltphysics for visionOS and bitcode is enabled. is there any way to disable bitcode?

What I have done

profile

[settings]
arch=armv8
build_type=Release
compiler=apple-clang
compiler.cppstd=gnu17
compiler.libcxx=libc++
compiler.version=15
os=Macos
[conf]
tools.cmake.cmaketoolchain:generator=Xcode
tools.apple:enable_bitcode=False

conanfile.py

from conan import ConanFile
from conan.tools.files import copy
from conan.tools.layout import basic_layout

class JoltPhysicsRecipe(ConanFile):
    settings = "os", "compiler", "build_type", "arch"

    def requirements(self):
        self.requires("joltphysics/3.0.1")

    def layout(self):
        basic_layout(self)

    def generate(self):
        for dep in self.dependencies.values():
            copy(self, "*.a", dep.cpp_info.libdirs[0], self.build_folder)
conan --version
Conan version 2.7.1
conan install . -s os=visionOS -s os.version=1.2 -s os.sdk=xros --build=missing
otool -l build-release/libJolt.a | grep bit-code | wc -l
    131

Have you read the CONTRIBUTING guide?

memsharded commented 2 months ago

Hi @shiena

Thanks for your question. Could you please clarify if the above profile is your default profile? So the settings defined with -s in command line will compose and overwrite that default profile? If not, maybe you are missing a -pr=myprofile argument?

What conan does with that conf defined is to define the following variables in conan_toolchain.cmake file:

set(CMAKE_XCODE_ATTRIBUTE_ENABLE_BITCODE "YES")
set(CMAKE_XCODE_ATTRIBUTE_BITCODE_GENERATION_MODE "bitcode")

Can you please verify, in the Conan cache folder where joltphysics is being built (you will see the folder in the output logs), that the conan_toolchain.cmake file there does not contain these variables defined?

If they are not, then it would be someone else, some other toolchain, script or configuration that is defining such value?

Do you have any link about the way to determine if a binary was built with bitcode, you use bit-code, but I see in some threads (maybe a bit old?) that they use a different __bitcode string to match, see for example otool -l binary_name | grep __bitcode

shiena commented 2 months ago

@memsharded I was not familiar with the use of profiles, so I had to specify parameters on the command line. I specified the following profiles, but the result was the same.

~/.conan2/profiles/visionOS

[settings]
arch=armv8
build_type=Release
compiler=apple-clang
compiler.cppstd=gnu17
compiler.libcxx=libc++
compiler.version=15
os=visionOS
os.version=1.2
os.sdk=xros
[conf]
tools.cmake.cmaketoolchain:generator=Xcode
tools.apple:enable_bitcode=False
conan install . -pr=visionOS --build=missing

build.log

% grep -i bit /Users/shiena/.conan2/p/b/joltp948e1b67e51b2/b/build/generators/conan_toolchain.cmake
# Define Apple architectures, sysroot, deployment target, bitcode, etc
set(BITCODE "")
# Bitcode OFF
set(CMAKE_XCODE_ATTRIBUTE_ENABLE_BITCODE "NO")
    string(APPEND CONAN_C_FLAGS " ${BITCODE} ${VISIBILITY} ${FOBJC_ARC}")
    string(APPEND CONAN_CXX_FLAGS " ${BITCODE} ${VISIBILITY} ${FOBJC_ARC}")

otool.log

memsharded commented 2 months ago

Interesting. I have no idea what could be happening, it seems Conan is doing the right thing, configuring bitcode to be disabled, still the build ignores it.

I don't have a Mac, so I'd need someone else from the team to check and investigate what could be happening.

shiena commented 2 months ago

@memsharded I don't know why, but I was able to disable bitcode by specifying build_type=Debug . If build_type=Release, bitcode will be enabled.

conan install . -pr visionOS --build=missing -s build_type=Debug
memsharded commented 2 months ago

I am curious, does it also happens with other libs? What if you create a lib with conan new cmake_lib -d name=mylib -d version=0.1, conan create . it, and then require it from your conanfile, does it have the same bitcode? I am trying to learn if it is specific to this one library, or is there something else in the tooling and Conan. Thanks!

shiena commented 2 months ago

I built and checked with imath/3.1.12 . It was possible to disable bitcode whether build_type was Release or Debug. So it seems to be a joltphysics specific issue.

memsharded commented 2 months ago

Very strange. I have checked the library source code and build scripts, and I cannot see anything that could be overwriting the bitcode definition.

You might try to do a verbose build for the built of joltphysics, activating the confs:

"tools.build:verbosity": "Verbosity of build systems if set. Possible values are 'quiet' and 'verbose'",
"tools.compilation:verbosity": "Verbosity of compilation tools if set. Possible values are 'quiet' and 'verbose'",

And we might have a look to the compilation commands of the different objects.

As I don't have a Mac, @uilianries might be able to do this on Monday, no rush. Thanks for the feedback!

shiena commented 2 months ago

The bitcode is disabled even if build_type is RelWithDebInfo or MinSizeRel. For some reason, bitcode is enabled only when build_type is Release.

shiena commented 2 months ago

The following is part of the release build log. The -flto=thin option appears only in release builds. So I feel that bitcode is enabled with this.

/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -DNDEBUG -I/Users/shiena/.conan2/p/b/joltp16ced3fe657d1/b/src/Build/.. -stdlib=libc++ -O3 -DNDEBUG -std=c++17 -flto=thin -arch arm64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/XROS.platform/Developer/SDKs/XROS1.2.sdk --target=arm64-apple-xros1.2 -fPIC -MD -MT CMakeFiles/Jolt.dir/Users/shiena/.conan2/p/b/joltp16ced3fe657d1/b/src/Jolt/Core/Color.cpp.o -MF CMakeFiles/Jolt.dir/Users/shiena/.conan2/p/b/joltp16ced3fe657d1/b/src/Jolt/Core/Color.cpp.o.d -o CMakeFiles/Jolt.dir/Users/shiena/.conan2/p/b/joltp16ced3fe657d1/b/src/Jolt/Core/Color.cpp.o -c /Users/shiena/.conan2/p/b/joltp16ced3fe657d1/b/src/Jolt/Core/Color.cpp
memsharded commented 2 months ago

Good research!

That would explain it. It is very likely that LTO relies on having the IR available (the bitcode), and that LTO only happens at Release configuration but not the others. Then, this would seem something specific to Jolt, see INTERPROCEDURAL_OPTIMIZATION option in https://github.com/jrouwe/JoltPhysics/blob/f95ad217f2c2797081cbc33b6d3463dd0ef65f84/Build/CMakeLists.txt#L29

Depending on what you need there could be different approaches:

shiena commented 2 months ago

Thank you very much. I will make a suggestion as the version pull reqs are out. https://github.com/conan-io/conan-center-index/pull/24670