Closed n-eq closed 3 years ago
Beginning of the CMakeLists.txt file where the dependencies are defined:
set(FCC_ANDROID_ARMV7_DEPENDENCIES log4c/3.00.09@fsv/android_armeabi-v7a RSARef2/1.00.02@fsv/android_armeabi-v7a)
set(FCC_ANDROID_ARMV8_DEPENDENCIES log4c/3.00.09@fsv/android_arm64-v8a RSARef2/1.00.02@fsv/android_arm64-v8a)
set(FCC_DEPENDENCIES log4c/3.00.09@fsv/stable RSARef2/1.00.02) # default
Hi @n-eq
I am trying to understand the issue and the code, a couple of quick questions trying to understand things:
log4c/3.00.09@fsv/android_armeabi-v7a
, it should typically be something like log4c/3.00.09@fsv/stable
. Conan is designed to handle the package binaries for multiple platforms and architectures transparently and uniformly in an integrated and built-in way. No need to call the packages after platform/arch, this would be an antipattern that most likely is making your code more complex and could be introducing issues and bugs.set(FCC_ANDROID_ARMV7_DEPENDENCIES log4c/3.00.09@fsv/android_armeabi-v7a RSARef2/1.00.02@fsv/android_armeabi-v7a)
? Sometimes it is fair to use something if the information was already there, but these are already Conan references, no something that existed before. It really makes things more complicated and unmanageable to put the dependencies information there, Conan has other more structured mechanisms, for example the conandata.yml
if you want to store them in a declarative way./Users/nabilelqatib/Library/Android/sdk/ndk/23.0.7599858/toolchains/llvm/prebuilt/darwin-x86_64/bin
? That variable could help to avoid typos and other problemsHi @memsharded again!
Thanks for the very quick reply.
- Why are you using the channel for defining the architecture?
log4c/3.00.09@fsv/android_armeabi-v7a
, it should typically be something likelog4c/3.00.09@fsv/stable
. Conan is designed to handle the package binaries for multiple platforms and architectures transparently and uniformly in an integrated and built-in way. No need to call the packages after platform/arch, this would be an antipattern that most likely is making your code more complex and could be introducing issues and bugs.
The first time I tried I noticed Conan didn't take the correct dependencies depending on the architecture, that's why I hardcoded them and never thought of it again really, might give it a quick try.
- Why do you define dependencies in your CMakeLists.txt, like
set(FCC_ANDROID_ARMV7_DEPENDENCIES log4c/3.00.09@fsv/android_armeabi-v7a RSARef2/1.00.02@fsv/android_armeabi-v7a)
? Sometimes it is fair to use something if the information was already there, but these are already Conan references, no something that existed before. It really makes things more complicated and unmanageable to put the dependencies information there, Conan has other more structured mechanisms, for example theconandata.yml
if you want to store them in a declarative way.
In fact the project I am working on consists of multiple modules using the same conanfile as a base, and this conanfile seeks the dependencies of each module in the CMakeLists file. I completely agree on the fact that it would have been better to define the dependencies in the conanfile, but for the moment my goal is to make the fewest modifications possible.
- Why not creating a variable in the profile file to store
/Users/nabilelqatib/Library/Android/sdk/ndk/23.0.7599858/toolchains/llvm/prebuilt/darwin-x86_64/bin
? That variable could help to avoid typos and other problems
Very good point. If you notice in the profile there's the line
PATH=[$android_ndk/toolchains/llvm/prebuilt/darwin-x86_64/bin]
which in theory exports the path to the binaries, but for some reason I ended up hardcoding them because the export was seemingly not working, might as well give it a new try.
The first time I tried I noticed Conan didn't take the correct dependencies depending on the architecture, that's why I hardcoded them and never thought of it again really, might give it a quick try.
Yes, please, it seems you are doing too much avoidable work, declaring dependencies should be really simpler.
I am also trying to see the output of
macro_name = "FCC_%s_%s_DEPENDENCIES" % (cross_build_settings[2], cross_build_settings[3])
ret = get_cmake_properties(macro_name.upper(), (), file_path)
print("get_dependencies %s: " % macro_name, ret)
In the log, but I don't see it. How is it possible that no prints are done? Could you introduce some more prints in the requirements()
and get_dependencies()
methods to try to understand what is happening?
I am also trying to see the output of
macro_name = "FCC_%s_%s_DEPENDENCIES" % (cross_build_settings[2], cross_build_settings[3]) ret = get_cmake_properties(macro_name.upper(), (), file_path) print("get_dependencies %s: " % macro_name, ret)
In the log, but I don't see it. How is it possible that no prints are done? Could you introduce some more prints in the
requirements()
andget_dependencies()
methods to try to understand what is happening?
Of course, I removed it in the first place from the logs but you can find it below.
def requirements(self):
for dep in get_dependencies(tools.get_cross_building_settings(self)):
print("Adding dependency: %s" % dep)
self.requires(dep)
def get_dependencies(cross_build_settings=None, file_path:str='CMakeLists.txt') -> List[str]:
if cross_build_settings is None:
dependencies = get_cmake_properties("FCC_DEPENDENCIES", (), file_path)
if os_is_linux():
os_dependencies = get_cmake_properties("FCC_LINUX_DEPENDENCIES", (), file_path)
elif os_is_Macos():
os_dependencies = get_cmake_properties("FCC_MAC_OS_DEPENDENCIES", (), file_path)
else:
os_dependencies = get_cmake_properties("FCC_WINDOWS_DEPENDENCIES", (), file_path)
deps = [complete_dependency(d) for d in dependencies + os_dependencies]
return deps
else:
macro_name = "FCC_%s_%s_DEPENDENCIES" % (cross_build_settings[2], cross_build_settings[3])
ret = get_cmake_properties(macro_name.upper(), (), file_path)
print("get_dependencies %s: " % macro_name, ret)
return ret
This is the output
Exporting package recipe
smc/3.00.02@sesamvitale/android_armeabi-v7a exports: Copied 1 '.txt' file: CMakeLists.txt
smc/3.00.02@sesamvitale/android_armeabi-v7a exports: Copied 32 '.cmake' files
smc/3.00.02@sesamvitale/android_armeabi-v7a exports: Copied 14 '.py' files
smc/3.00.02@sesamvitale/android_armeabi-v7a exports_sources: Copied 82 '.h' files
smc/3.00.02@sesamvitale/android_armeabi-v7a exports_sources: Copied 53 '.c' files
smc/3.00.02@sesamvitale/android_armeabi-v7a exports_sources: Copied 1 '.cpp' file: smc.cpp
smc/3.00.02@sesamvitale/android_armeabi-v7a exports_sources: Copied 2 '.def' files: smc64.def, smc32.def
smc/3.00.02@sesamvitale/android_armeabi-v7a exports_sources: Copied 1 '.rc' file: smc.rc
smc/3.00.02@sesamvitale/android_armeabi-v7a exports_sources: Copied 1 '.plist' file: Info.plist
smc/3.00.02@sesamvitale/android_armeabi-v7a exports_sources: Copied 1 file: smc_export_script
smc/3.00.02@sesamvitale/android_armeabi-v7a: The stored package has not changed
smc/3.00.02@sesamvitale/android_armeabi-v7a: Exported revision: 84d620a12e3253829140d1bf3ca5fd46
Configuration (profile_host):
[settings]
arch=armv7
build_type=Release
compiler=clang
compiler.libcxx=libc++
compiler.version=12
os=Android
os.api_level=22
[options]
shared=False
smc:shared=False
[build_requires]
[env]
AR=llvm-ar
AS=arm-linux-androideabi-as
CC=armv7a-linux-androideabi22-clang
CHOST=armv7a-linux-androideabi
CONAN_CMAKE_TOOLCHAIN_FILE=/Users/n-eq/Library/Android/sdk/ndk/23.0.7599858/build/cmake/android.toolchain.cmake
CXX=armv7a-linux-androideabi22-clang++
LD=armv7a-linux-androideabi-ld
PATH=[/Users/n-eq/Library/Android/sdk/ndk/23.0.7599858/toolchains/llvm/prebuilt/darwin-x86_64/bin]
RANLIB=llvm-ranlib
STRIP=llvm-strip
Configuration (profile_build):
[settings]
arch=x86_64
arch_build=x86_64
build_type=Release
compiler=apple-clang
compiler.libcxx=libc++
compiler.version=11.0
os=Macos
os_build=Macos
[options]
[build_requires]
[env]
get_dependencies FCC_Android_armv7_DEPENDENCIES: ('log4c/3.00.09@sesamvitale/android_armeabi-v7a', 'RSARef2/1.00.02@sesamvitale/android_armeabi-v7a')
Adding dependency: log4c/3.00.09@sesamvitale/android_armeabi-v7a
Adding dependency: RSARef2/1.00.02@sesamvitale/android_armeabi-v7a
get_dependencies FCC_Android_armv7_DEPENDENCIES: ('log4c/3.00.09@sesamvitale/android_armeabi-v7a', 'RSARef2/1.00.02@sesamvitale/android_armeabi-v7a')
Adding dependency: log4c/3.00.09@sesamvitale/android_armeabi-v7a
Adding dependency: RSARef2/1.00.02@sesamvitale/android_armeabi-v7a
JSON file created at '/Users/n-eq/Dev/smc/__liv/pkg_info/pkg_info_3.00.02_android_armeabi-v7a_Release_static.json'
ERROR: Loop detected in context host: 'log4c/3.00.09@sesamvitale/android_armeabi-v7a' requires 'log4c/3.00.09@sesamvitale/android_armeabi-v7a' which is an ancestor too
Cheers,
I am not sure why you are getting 2 identical get_dependencies()
printed. Maybe adding the current package name, something like:
def requirements(self):
print("Computing the requirements of {}/{}".format(self.name, self.version))
for dep in get_dependencies(tools.get_cross_building_settings(self)):
print("Adding dependency: %s" % dep)
self.requires(dep)
Sure @memsharded , here are the lines it outputs:
Computing the requirements of smc/3.00.02
get_dependencies FCC_Android_armv7_DEPENDENCIES: ('log4c/3.00.09@sesamvitale/android_armeabi-v7a', 'RSARef2/1.00.02@sesamvitale/android_armeabi-v7a')
Adding dependency: log4c/3.00.09@sesamvitale/android_armeabi-v7a
Adding dependency: RSARef2/1.00.02@sesamvitale/android_armeabi-v7a
Computing the requirements of log4c/3.00.09
get_dependencies FCC_Android_armv7_DEPENDENCIES: ('log4c/3.00.09@sesamvitale/android_armeabi-v7a', 'RSARef2/1.00.02@sesamvitale/android_armeabi-v7a')
Adding dependency: log4c/3.00.09@sesamvitale/android_armeabi-v7a
Adding dependency: RSARef2/1.00.02@sesamvitale/android_armeabi-v7a
It seems to confirm log4c
depends on itself...
When building log4c
and RSARef2
independently:
Computing the requirements of log4c/3.00.09
get_dependencies FCC_Android_armv7_DEPENDENCIES: ()
Computing the requirements of RSARef2/1.00.02
get_dependencies FCC_Android_armv7_DEPENDENCIES: ()
Yes, exactly, that is the key:
Computing the requirements of log4c/3.00.09
get_dependencies FCC_Android_armv7_DEPENDENCIES: ('log4c/3.00.09@sesamvitale/android_armeabi-v7a', 'RSARef2/1.00.02@sesamvitale/android_armeabi-v7a')
Adding dependency: log4c/3.00.09@sesamvitale/android_armeabi-v7a
Adding dependency: RSARef2/1.00.02@sesamvitale/android_armeabi-v7a
the log4c
is requiring itself, as a result of the evaluation of get_dependencies()
.
Can we make sure what is in the CMakeLists.txt
of the log4c
folder/recipe? If it is different, then I'd suggest converting to absolute path file_path
inside your method and printing it too. It could be that you are reading a different CMakeLists.txt
file that the one you intend. You probably can use something like r = load(os.path.join(self.recipe_folder, "file.txt"))
to make the path relative to the current conanfile.py.
Also, I'd like to insist, these issues are difficult to debug because of the complexity of the system, I'd really suggest to put in your future plans
Yes, exactly, that is the key:
Computing the requirements of log4c/3.00.09 get_dependencies FCC_Android_armv7_DEPENDENCIES: ('log4c/3.00.09@sesamvitale/android_armeabi-v7a', 'RSARef2/1.00.02@sesamvitale/android_armeabi-v7a') Adding dependency: log4c/3.00.09@sesamvitale/android_armeabi-v7a Adding dependency: RSARef2/1.00.02@sesamvitale/android_armeabi-v7a
the
log4c
is requiring itself, as a result of the evaluation ofget_dependencies()
.Can we make sure what is in the
CMakeLists.txt
of thelog4c
folder/recipe? If it is different, then I'd suggest converting to absolute pathfile_path
inside your method and printing it too. It could be that you are reading a differentCMakeLists.txt
file that the one you intend. You probably can use something liker = load(os.path.join(self.recipe_folder, "file.txt"))
to make the path relative to the current conanfile.py.
You're absolutely right. For an (yet!) unknown reason log4c uses smc's CMakeLists.txt file. Again I'm not really allowed to make new architectural choices (the goal of the project was to cross-build the packages to Android.) but I perfectly understand your point. I'll try your suggestion but I think we have found the cause of the problem.
Also, I'd like to insist, these issues are difficult to debug because of the complexity of the system, I'd really suggest to put in your future plans
- Simplify the definitions of dependencies, moving them to conanfile.py or conandata.yml
- Simplify the architecture management, making channels always "stable", and letting Conan manage the different binaries. I think this can make you happier π
Many thanks for the suggestions. I'll try to set the channels as "stable" as soon as I figure out a way to cross-build seamlessly.
Great! I think the reason is that Conan uses the current directory if not specified for some operations, and it is necessary to explicitly use the self.recipe_folder
. If you applied it and it worked, then I think we can close the issue, feel free to close it, and if you need further help in the future for the other comments, you can always open a new ticket. Thanks!
Closing the issue now. Thanks @memsharded, I appreciate your time and kindness.
Hi,
(This ticket is a follow-up on the following question).
I am trying to compile a package (smc) which has two dependencies (RSARef2 and log4c), the three packages live in the same project and their source code is provided (Note: I cannot share the source code files for intellectual property reasons, but it mainly consists of C code). So far I've been able to compile for MacOS and Windows, but when trying to compile for Android targets (Armv7 for example), the process fails with the following error (see full log below):
ERROR: Loop detected in context host: 'log4c/3.00.09@fsv/android_armeabi-v7a' requires 'log4c/3.00.09@fsv/android_armeabi-v7a' which is an ancestor too
Environment Details (include every applicable attribute)
Steps to reproduce (Include if Applicable)
For this example I am cross-compiling for Android armv7 targets, using the following profiles and command.
[settings] arch=armv7 build_type=Release compiler=clang compiler.libcxx=libc++ compiler.version=12 os=Android os.api_level=$api_level [build_requires] [options] [env] PATH=[$android_ndk/toolchains/llvm/prebuilt/darwin-x86_64/bin] CHOST=$target_host AR=/Users/nabilelqatib/Library/Android/sdk/ndk/23.0.7599858/toolchains/llvm/prebuilt/darwin-x86_64/bin/llvm-ar AS=/Users/nabilelqatib/Library/Android/sdk/ndk/23.0.7599858/toolchains/llvm/prebuilt/darwin-x86_64/bin/arm-linux-androideabi-as RANLIB=/Users/nabilelqatib/Library/Android/sdk/ndk/23.0.7599858/toolchains/llvm/prebuilt/darwin-x86_64/bin/llvm-ranlib CC=/Users/nabilelqatib/Library/Android/sdk/ndk/23.0.7599858/toolchains/llvm/prebuilt/darwin-x86_64/bin/armv7a-linux-androideabi22-clang CXX=/Users/nabilelqatib/Library/Android/sdk/ndk/23.0.7599858/toolchains/llvm/prebuilt/darwin-x86_64/bin/armv7a-linux-androideabi22-clang++ LD=$target_host-ld STRIP=/Users/nabilelqatib/Library/Android/sdk/ndk/23.0.7599858/toolchains/llvm/prebuilt/darwin-x86_64/bin/llvm-strip CONAN_CMAKE_TOOLCHAIN_FILE=$android_ndk/build/cmake/android.toolchain.cmake
[settings] os=Macos os_build=Macos arch=x86_64 arch_build=x86_64 compiler=apple-clang compiler.version=11.0 compiler.libcxx=libc++ build_type=Release [options] [build_requires] [env]
Logs (Executed commands with output) (Include/Attach if Applicable)
I appreciate your help. (cc @memsharded)