search

eclipse-cyclonedds / cyclonedds

Eclipse Cyclone DDS project
https://projects.eclipse.org/projects/iot.cyclonedds
Other
864 stars 354 forks source link

Can't find native ddsconf when cross compiling for BBB using buildroot #1075

Closed pscheie closed 2 years ago

pscheie commented 2 years ago

I'm trying to install cyclonedds into an image for the BeagleBone Black, using buildroot on my laptop (x86_64). So, that means a cross compile. I'm getting the error "CMake Error at src/tools/ddsconf/CMakeLists.txt:64 (message): Could not find a native ddsconf executable, please make sure it is available on PATH" What puzzles me is line 64 in CMakeLists.txt is in the else() section of 'if (CMAKE_CROSSCOMPILING)', meaning CMake thinks it's not cross compiling. Not sure if this is related to #967 or not; that was the only message I could find that was related.

BTW, is this the best place to ask questions? I saw there's a mailing list but the last messages on it were last summer. Is there an IRC channel for cyclonedds?

pscheie commented 2 years ago

Just found #794, which sounds like the same problem I have. I do have cyclonedds already installed on my host system; how do I set CMAKE_CROSSCOMPILING to true, and give cmake the path to my native cyclonedds installation?

pscheie commented 2 years ago

I added the cyclonedds bin/ directory to my $PATH and then ran buildroot's make again (which calls CMake for cyclonedds, etc), and got a bit further. Now I'm getting this error:

[ 30%] Building C object src/core/CMakeFiles/ddsc.dir/ddsi/src/q_xevent.c.o [ 30%] Building C object src/core/CMakeFiles/ddsc.dir/ddsi/src/q_xmsg.c.o [ 31%] Building C object src/core/CMakeFiles/ddsc.dir/ddsi/src/q_freelist.c.o [ 31%] Building C object src/core/CMakeFiles/ddsc.dir/ddsi/src/sysdeps.c.o [ 32%] Building C object src/core/CMakeFiles/ddsc.dir/ddsi/src/ddsi_lifespan.c.o [ 32%] Building C object src/core/CMakeFiles/ddsc.dir/ddsi/src/ddsi_security_msg.c.o /home/petre/buildroot-2021.02.2.beaglebone/output/build/cyclonedds-0.8.1/src/core/ddsi/src/sysdeps.c:30:10: fatal error: execinfo.h: No such file or directory 30 | #include | ^~~~ compilation terminated.

Indeed, there is no execinfo.h anywhere in buildroot's cyclonedds build directory. Any suggestions?

pscheie commented 2 years ago

Update: I switched from uclibc to glibc in buildroot's toolchain because uclibc doesn't provide execinfo.h apparently (?). With that change, it now builds. I notice in the target output that ddsconf and idlc are not installed; is that noteworthy?

eboasson commented 2 years ago

Regarding the intricacies of a cross-build, I can't tell you much (as I am sure you also noted on the related issues you found), unlike some other people. But I may be able to help on some other points:

BTW, is this the best place to ask questions? I saw there's a mailing list but the last messages on it were last summer. Is there an IRC channel for cyclonedds?

This is indeed probably the best place to ask, not because the mailing list is ignored but because keeping things on GitHub probably makes it easiest for other people to find related things. There is gitter as well, but the response times for an initial question are usually not much better than on GitHub because there's not enough traffic to keep a window open — but of course it works better for interaction.

Indeed, there is no execinfo.h anywhere in buildroot's cyclonedds build directory. Any suggestions?

Update: I switched from uclibc to glibc in buildroot's toolchain because uclibc doesn't provide execinfo.h apparently (?). With that change, it now builds.

I like that you solved the problem, but switching to glibc seems like a heavyweight solution. execinfo.h is really only used for backtrace, which in turn is really only used to provide some additional information when the "thread liveliness monitoring" option is enabled and some thread fails to make progress as expected. So the alternative solution would've been to use the C preprocessor to leave out that bit of code if on uclibc. If you know the conditions involved, let's add a check.

I notice in the target output that ddsconf and idlc are not installed; is that noteworthy?

ddsconf is a build-time tool, it takes the configuration tables in src/core/ddsi/include/dds/ddsi/ddsi_cfgelems.h and generates

The first three are only optionally generated (with BUILD_DOCS) because the output is committed in the repo as it is convenient to have them available in GitHub to link to and it changes rarely anyway. The last one however is needed during the build. But it is useless for anyone using Cyclone.

idlc isn't installed in a cross-build on the assumption that the you'd be running the IDL compiler on the host for development — so for the typical embedded software development.

So those are both as expected.

k0ekk0ek commented 2 years ago

Hi @pscheie, @eboasson already gave an excellent answer wrt the how and what, so I'll just stick to providing quick answers to your questions. CMAKE_CROSSCOMPILING is set to true automatically when using a toolchain file. You can configure CMake to look for host binaries in that file as well as specify many other settings. CMAKE_FIND_ROOT_PATH_MODE_PROGRAM is the option you're looking for in this case. As for execinfo.h, that header (as @eboasson indicated) is only required if you want to print stack traces. sysdeps.c contains the following preprocessor check (defined __linux && (defined __GLIBC__ || defined __UCLIBC__))) (among others). That probably means __linux is defined in your scenario(?) Depending on the operating system you're cross compiling for, that may or may not be correct.

You can probably work around this by disabling that piece of code for now if you want to use uclibc(?)

For future cases we probably want to move the logging of stack traces to the os abstraction level and check for availability of that header etc.

pscheie commented 2 years ago

Hi @k0ekk0ek and @eboasson, thanks for the replies. Based on what you've said, I don't think I need execinfo.h. Regarding @eboasson 's question:

"So the alternative solution would've been to use the C preprocessor to leave out that bit of code if on uclibc. If you know the conditions involved, let's add a check."

Buildroot has a couple of variables stored in the .config file, BR2_TOOLCHAIN_BUILDROOT_UCLIBC and BR2_TOOLCHAIN_BUILDROOT_GLIBC, and whichever is set to Y is used (there's also one for MUSL). UCLIBC is the default. That variable is set in its environment, and I can add some logic in the BR config to pass a value as an argument to CMake. But I don't know what needs to change for CMake to use that value (something in CMakeLists.txt, I presume).

I can live with using glibc, but doing so does triple the size of my rootfs, and I'd rather fix the problem properly which then gives me the benefit of sticking with uclibc.

thijsmie commented 2 years ago

Hi @pscheie! We've just merged a ddsconf rework on master so it is not needed at compile time if there are no changes in the source (the files are hashed and checked into source control). Can you let us know if this indeed makes the process easier, as we hope it does? You should not need a 'host cyclonedds' while crosscompiling now.

pscheie commented 2 years ago

Hi @thijsmie, Glad to hear that! Yes, I'll test it out over the next couple of days. Thanks to everyone for all the effort.

pscheie commented 2 years ago

Sorry about the delay in reporting back; I got pulled off to another task for a bit. I'm getting an error but I may have misunderstood what to expect from this latest merge. During configuration, it (make) is complaining that it can't find idlc. I have removed (renamed) the cyclonedds directory on my build host because I interpreted

You should not need a 'host cyclonedds' while crosscompiling now.

to mean I didn't need cyclonedds installed on the build host; but perhaps that's not the case?

thijsmie commented 2 years ago

Hi @pscheie!

Right, I forgot you need the idl compiler, but only for a very specific part: ddsperf. If you do not need ddsperf on your target platform you can pass BUILD_DDSPERF=OFF to cmake and then you should be able to build without host cyclone.

pscheie commented 2 years ago

I don't know if I need ddsperf or not (what does it do?), but in the interest of testing out the latest commits, I set -DBUILD_DDSPERF=OFF in the package config file for buildroot. But it's still failing at the same point, complaining about not finding idlc: ``

cyclonedds master Extracting gzip -d -c /home/petre/buildroot-2021.02.8.beaglebone/dl/cyclonedds/cyclonedds-master-br1.tar.gz | tar --strip-components=1 -C /home/petre/buildroot-2021.02.8.beaglebone/output/build/cyclonedds-master -xf - cyclonedds master Patching cyclonedds master Configuring (mkdir -p /home/petre/buildroot-2021.02.8.beaglebone/output/build/cyclonedds-master/ && cd /home/petre/buildroot-2021.02.8.beaglebone/output/build/cyclonedds-master/ && rm -f CMakeCache.txt && PATH="/home/petre/buildroot-2021.02.8.beaglebone/output/host/bin:/home/petre/buildroot-2021.02.8.beaglebone/output/host/sbin:/opt/cmake-3.18.6/bin:/home/petre/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/opt/cyclonedds/bin" /opt/cmake-3.18.6/bin/cmake /home/petre/buildroot-2021.02.8.beaglebone/output/build/cyclonedds-master/ -DCMAKE_TOOLCHAIN_FILE="/home/petre/buildroot-2021.02.8.beaglebone/output/host/share/buildroot/toolchainfile.cmake" -DCMAKE_INSTALL_PREFIX="/usr" -DCMAKE_COLOR_MAKEFILE=OFF -DBUILD_DOC=OFF -DBUILD_DOCS=OFF -DBUILD_EXAMPLE=OFF -DBUILD_EXAMPLES=OFF -DBUILD_TEST=OFF -DBUILD_TESTS=OFF -DBUILD_TESTING=OFF -DBUILD_SHARED_LIBS=ON -DCMAKE_INSTALL_PREFIX= -DBUILD_EXAMPLES=ON -DBUILD_DDSPERF=OFF ) -- The C compiler identification is GNU 9.4.0 -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working C compiler: /home/petre/buildroot-2021.02.8.beaglebone/output/host/bin/arm-buildroot-linux-gnueabihf-gcc - skipped -- Detecting C compile features -- Detecting C compile features - done -- Looking for getopt.h -- Looking for getopt.h - found -- Building with OpenSSL support -- Looking for gethostname -- Looking for gethostname - found -- Looking for inet_ntop -- Looking for inet_ntop - found -- Looking for inet_pton -- Looking for inet_pton - found -- Looking for getaddrinfo -- Looking for getaddrinfo - found -- Looking for gethostbyname_r -- Looking for gethostbyname_r - found -- Looking for sys/types.h -- Looking for sys/types.h - found -- Looking for stdint.h -- Looking for stdint.h - found -- Looking for stddef.h -- Looking for stddef.h - found -- Check size of struct sockaddr_in6 -- Check size of struct sockaddr_in6 - done -- Looking for pthread.h -- Looking for pthread.h - found -- Performing Test CMAKE_HAVE_LIBC_PTHREAD -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed -- Looking for pthread_create in pthreads -- Looking for pthread_create in pthreads - not found -- Looking for pthread_create in pthread -- Looking for pthread_create in pthread - found -- Found Threads: TRUE
-- Looking for clock_gettime in c -- Looking for clock_gettime in c - found -- Performing Test COMPILER_HAS_HIDDEN_VISIBILITY -- Performing Test COMPILER_HAS_HIDDEN_VISIBILITY - Success -- Performing Test COMPILER_HAS_HIDDEN_INLINE_VISIBILITY -- Performing Test COMPILER_HAS_HIDDEN_INLINE_VISIBILITY - Failed -- Performing Test COMPILER_HAS_DEPRECATED_ATTR -- Performing Test COMPILER_HAS_DEPRECATED_ATTR - Success CMake Error at cmake/Modules/Generate.cmake:43 (find_program): Could not find _idlc_executable using the following names: idlc Call Stack (most recent call first): cmake/Modules/Generate.cmake:31 (idlc_generate_generic) examples/helloworld/CMakeLists.txt:24 (idlc_generate) -- Configuring incomplete, errors occurred! See also "/home/petre/buildroot-2021.02.8.beaglebone/output/build/cyclonedds-master/CMakeFiles/CMakeOutput.log". See also "/home/petre/buildroot-2021.02.8.beaglebone/output/build/cyclonedds-master/CMakeFiles/CMakeError.log". make[1]: *** [package/pkg-generic.mk:241: /home/petre/buildroot-2021.02.8.beaglebone/output/build/cyclonedds-master/.stamp_configured] Error 1

You can see BUILD_DDSPERF=OFF is being passed to cmake at the top, but it's as if it's being ignored. Suggestions?

thijsmie commented 2 years ago

ddsperf is a tool to measure througput, latency and other things while using different datatypes for topics. I would say if you are simply deploying an application it is usually not relevant, but if you want to set up a developer environment it is a useful tool

The cause of your current error was luckily caught in the log you posted: examples/helloworld/CMakeLists.txt:24 (idlc_generate). The CycloneDDS examples rely on an idl compiler being available, the types they rely on are included in the source tree as .idl files. If you want to BUILD_EXAMPLES=ON then you'll have to have a host CycloneDDS to provide the idl compiler as before. Performing a similar trick for idlc as we did for ddsconf to supply a host executable in a target environment would require a lot of CMake fiddling and would probably be pretty fragile, so I don't see it happening (anytime soon).

pscheie commented 2 years ago

Ah, of course. It's obvious now that you say it. Ironically, I'd been looking at the BUILD_EXAMPLES parameter and thinking I should turn that off because I don't need them on the target; but then I'd figure I'll just leave it for now. Serves me right for being lazy and untidy.

So, cyclonedds does now build for the target without needing to have cyclonedds installed on the host--yay! The not-so-good news is that the python-cyclonedds package is failing to build. But I'll take that up on the cyclonedds-python github page.

Thanks for the simplification of cross-compiling cyclonedds.

pscheie commented 2 years ago

Just a follow-up to my previous message: cyclonedds and cyclonedds-python both build (cross-compile) successfully now. The problem with cyclonedds-python was caused by my pulling an older commit.

thijsmie commented 2 years ago

All problems with ddsconf are behind us with the slightly complicated new _confgen cachable generation, that I am sure will never turn up any issues again :roll_eyes:.

alexleel commented 1 year ago

ddsperf is a tool to measure througput, latency and other things while using different datatypes for topics. I would say if you are simply deploying an application it is usually not relevant, but if you want to set up a developer environment it is a useful tool

The cause of your current error was luckily caught in the log you posted: examples/helloworld/CMakeLists.txt:24 (idlc_generate). The CycloneDDS examples rely on an idl compiler being available, the types they rely on are included in the source tree as .idl files. If you want to BUILD_EXAMPLES=ON then you'll have to have a host CycloneDDS to provide the idl compiler as before. Performing a similar trick for idlc as we did for ddsconf to supply a host executable in a target environment would require a lot of CMake fiddling and would probably be pretty fragile, so I don't see it happening (anytime soon).

Hi @thijsmie I set BUILD_EXAMPLES=ON to build some example, however, some errors happend as following: ..Could not find _idlc_executable using the following names: idlc and the releated file Generate.cmake :

if(CMAKE_CROSSCOMPILING) find_program(_idlc_executable idlc NO_CMAKE_FIND_ROOT_PATH REQUIRED)

if(_idlc_executable)
  set(_idlc_depends "")
else()
  message(FATAL_ERROR "Cannot find idlc executable")
endif()

What can I do to debug the issue? Thanks.