Closed olipinski closed 4 years ago
And then you build bitbake xfstk
or bitbake xfstk_native
? I think when I have target set to edison xfstk will run on edison but won't on host.
I have tried both bitbake xfstk
and bitbake xfstk-native
, but they both build for edison. So that is something that I will look at how to fix.
Also most of the dependencies will not be resolved, so I am not sure what to do with that.
I have tried both
bitbake xfstk
andbitbake xfstk-native
, but they both build for edison. So that is something that I will look at how to fix.
I get errors. And I believe we will be needing xfstk-native
as xfstk
will be built to run on edison itself.
Also most of the dependencies will not be resolved, so I am not sure what to do with that.
When I look for a Yocto package, I first look up the original Ubuntu package here: https://packages.ubuntu.com/ then goto Yocto layers https://layers.openembedded.org/layerindex/branch/zeus/recipes/ to find equivalent.
I have tried both
bitbake xfstk
andbitbake xfstk-native
, but they both build for edison. So that is something that I will look at how to fix.I get errors. And I believe we will be needing
xfstk-native
asxfstk
will be built to run on edison itself.
I had errors on both too, as the qt5 is missing, but this is the build configuration for xfstk-native
Build Configuration:
BB_VERSION = "1.44.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "ubuntu-20.04"
TARGET_SYS = "i686-poky-linux"
MACHINE = "edison"
DISTRO = "poky-edison"
DISTRO_VERSION = "3.0.4"
TUNE_FEATURES = "m32 corei7"
TARGET_FPU = ""
It looks like it targets i686 even for -native
.
Also most of the dependencies will not be resolved, so I am not sure what to do with that.
When I look for a Yocto package, I first look up the original Ubuntu package here: https://packages.ubuntu.com/ then goto Yocto layers https://layers.openembedded.org/layerindex/branch/zeus/recipes/ to find equivalent.
Thank you for the advice! That will make it easier to find the dependencies!
I've now added the meta-qt5 to setup.sh and it configures up to a point, but there are still some issues wth qmake.
I added the following to the layer.conf:
PACKAGECONFIG_append_pn-qtbase-native = " gui"
And to recipe:
inherit cmake qmake5_paths
DEPENDS = "libxml2-native libusb-native boost-native qtbase-native"
EXTRA_OECMAKE += " -DWARNING_AS_ERROR=OFF -DOE_QMAKE_PATH_EXTERNAL_HOST_BINS='${OE_QMAKE_PATH_EXTERNAL_HOST_BINS}'"
OECMAKE_TARGET_COMPILE = "xfstk-dldr-solo"
BUILD_VERSION = "${PV}"
We'd probably want to rename the recipe to ..1.8.3
Now it configures and starts compiling. Eventually it fails with:
recipe-sysroot-native/usr/include/QtCore/qprocess.h:43:10: fatal error: QtCore/qiodevice.h: No such file or directory
43 | #include <QtCore/qiodevice.h>
| ^~~~~~~~~~~~~~~~~~~~
compilation terminated.
but I don't see why.
That is interesting. I have added the same lines (also have now pushed them as it goes further along) but for me the compilation stops on doxygen when it tries to configure it.
ERROR: xfstk-native-1.8.3-r0 do_configure: Execution of '/home/vagrant/test/out/linux64/build/tmp/work/x86_64-linux/xfstk-native/1.8.3-r0/temp/run.do_configure.76305' failed with exit code 1:
**** Starting Downloader Build Configuration ****
-- The CXX compiler identification is GNU 9.3.0
-- Check for working CXX compiler: /home/vagrant/test/out/linux64/build/tmp/hosttools/g++
-- Check for working CXX compiler: /home/vagrant/test/out/linux64/build/tmp/hosttools/g++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Checking C++11 support for "auto" (N2546)
-- Checking C++11 support for "auto" (N2546) -- works
-- Checking C++11 support for "nullptr" (N2431)
-- Checking C++11 support for "nullptr" (N2431) -- works
-- Checking C++11 support for "lambda" (N2927)
-- Checking C++11 support for "lambda" (N2927) -- works
-- Checking C++11 support for "static_assert" (N1720)
-- Checking C++11 support for "static_assert" (N1720) -- works
-- Checking C++11 support for "rvalue_references" (N2118)
-- Checking C++11 support for "rvalue_references" (N2118) -- works
-- Checking C++11 support for "decltype" (N2343)
-- Checking C++11 support for "decltype" (N2343) -- works
-- Checking C++11 support for "cstdint"
-- Checking C++11 support for "cstdint" -- works
-- Checking C++11 support for "long_long" (N1811)
-- Checking C++11 support for "long_long" (N1811) -- works
-- Checking C++11 support for "variadic_templates" (N2555)
-- Checking C++11 support for "variadic_templates" (N2555) -- works
-- Checking C++11 support for "constexpr" (N2235)
-- Checking C++11 support for "constexpr" (N2235) -- works
-- Checking C++11 support for "sizeof_member" (N2253)
-- Checking C++11 support for "sizeof_member" (N2253) -- works
-- Checking C++11 support for "__func__" (N2340)
-- Checking C++11 support for "__func__" (N2340) -- works
-- The following C++ features are required:
++ Build version is 1.8.3 ++
-- Defaulting to release build (use -DCMAKE_BUILD_TYPE:STRING=Debug for debug build)
-- Could NOT find CCache (missing: CCACHE)
**** Configuring command-line-helper ****
-- Found Boost: /home/vagrant/test/out/linux64/build/tmp/work/x86_64-linux/xfstk-native/1.8.3-r0/recipe-sysroot-native/usr/include (found version "1.71.0") found components: program_options
**** Configuring Factory ****
-- Found USB: /home/vagrant/test/out/linux64/build/tmp/work/x86_64-linux/xfstk-native/1.8.3-r0/recipe-sysroot-native/usr/lib/libusb.so
**** Configuring Downloader Solo ****
-- Found Boost: /home/vagrant/test/out/linux64/build/tmp/work/x86_64-linux/xfstk-native/1.8.3-r0/recipe-sysroot-native/usr/lib/cmake/Boost-1.71.0/BoostConfig.cmake (found version "1.71.0") found components: program_options
**** Configuring API ****
-- Found USB: /home/vagrant/test/out/linux64/build/tmp/work/x86_64-linux/xfstk-native/1.8.3-r0/recipe-sysroot-native/usr/lib/libusb.so
**** Configuring downloader-ui ****
**** Configuring Plugin: Cloverview ****
**** Configuring Plugin: CloverviewPlus ****
**** Configuring Plugin: Medfield ****
**** Configuring Merrifield Plugin ****
**** Configuring Baytrail Plugin ****
**** Begin Generate documenation ****
-- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE)
CMake Error at ancillary/cmake_modules/GenerateDocumentation.cmake:22 (MESSAGE):
Please install Doxygen.
Call Stack (most recent call first):
CMakeLists.txt:101 (INCLUDE)
-- Configuring incomplete, errors occurred!
See also "/home/vagrant/test/out/linux64/build/tmp/work/x86_64-linux/xfstk-native/1.8.3-r0/build/CMakeFiles/CMakeOutput.log".
WARNING: exit code 1 from a shell command.
When I added doxygen now it gives the weirdest error - no Makefile??
ERROR: xfstk-native-1.8.3-r0 do_compile: Execution of '/home/vagrant/test/out/linux64/build/tmp/work/x86_64-linux/xfstk-native/1.8.3-r0/temp/run.do_compile.76203' failed with exit code 2:
make: *** No targets specified and no makefile found. Stop.
WARNING: exit code 2 from a shell command.
ERROR: Logfile of failure stored in: /home/vagrant/test/out/linux64/build/tmp/work/x86_64-linux/xfstk-native/1.8.3-r0/temp/log.do_compile.76203
Log data follows:
| DEBUG: Executing shell function do_compile
| make: *** No targets specified and no makefile found. Stop.
| WARNING: exit code 2 from a shell command.
| ERROR: Execution of '/home/vagrant/test/out/linux64/build/tmp/work/x86_64-linux/xfstk-native/1.8.3-r0/temp/run.do_compile.76203' failed with exit code 2:
| make: *** No targets specified and no makefile found. Stop.
| WARNING: exit code 2 from a shell command.
ERROR: Task (virtual:native:/home/vagrant/test/meta-intel-edison/meta-xfstk/recipes-xfstk/xfstk/xfstk_1.8.3.bb:do_compile) failed with exit code '1'
NOTE: Tasks Summary: Attempted 329 tasks of which 272 didn't need to be rerun and 1 failed.
Summary: 1 task failed:
virtual:native:/home/vagrant/test/meta-intel-edison/meta-xfstk/recipes-xfstk/xfstk/xfstk_1.8.3.bb:do_compile
So I cannot reproduce the error :
recipe-sysroot-native/usr/include/QtCore/qprocess.h:43:10: fatal error: QtCore/qiodevice.h: No such file or directory 43 | #include <QtCore/qiodevice.h> | ^~~~~~~~~~~~~~~~~~~~ compilation terminated.
as my compilation either skips that or doesn't get that far?
Oh, sorry, yes, I also removed your do_compile() lines. Now it builds OECMAKE_TARGET_COMPILE = "xfstk-dldr-solo"
so no docs.
Oh, sorry, yes, I also removed your do_compile() lines. Now it builds
OECMAKE_TARGET_COMPILE = "xfstk-dldr-solo"
so no docs.
Oh, right! I have removed that now and it does go through slightly. But my compilation now fails on libboost for some reason:
| /home/vagrant/test/out/linux64/build/tmp/work/x86_64-linux/xfstk-native/1.8.3-r0/git/xfstk-sources/core/factory/./platforms/baytrail/baytrailoptions.h:23:10: fatal error: boost/program_options.hpp: No such file or directory
| 23 | #include <boost/program_options.hpp>
| | ^~~~~~~~~~~~~~~~~~~~~~~~~~~
| compilation terminated.
Also it still seems that doxygen is required on my machine, at least when I remove it I get the same errors as before.
Did you add the following to the layer.conf:
PACKAGECONFIG_append_pn-qtbase-native = " gui"
Yup, that's in https://github.com/edison-fw/meta-intel-edison/pull/116/commits/ea04b2f6c4c30eebdc3b3f7d87b75dca4ad5f5b2 . It should now be exactly the same, but somehow is not.
Maybe because you don't have -native
in DEPENDS = "libxml2 libusb boost qtbase doxygen"
?
I have tried with a -native
but I'm still getting the error as below:
/factory/./platforms/baytrail/baytrailoptions.h:23:10: fatal error: boost/program_options.hpp: No such file or directory
| 23 | #include <boost/program_options.hpp>
| | ^~~~~~~~~~~~~~~~~~~~~~~~~~~
| compilation terminated.
| ninja: build stopped: subcommand failed.
| WARNING: exit code 1 from a shell command.
|
ERROR: Task (virtual:native:/home/vagrant/test/meta-intel-edison/meta-xfstk/recipes-xfstk/xfstk/xfstk_1.8.3.bb:do_compile) failed with exit code '1'
NOTE: Tasks Summary: Attempted 329 tasks of which 328 didn't need to be rerun and 1 failed.
Summary: 1 task failed:
virtual:native:/home/vagrant/test/meta-intel-edison/meta-xfstk/recipes-xfstk/xfstk/xfstk_1.8.3.bb:do_compile
Summary: There was 1 WARNING message shown.
Summary: There was 1 ERROR message shown, returning a non-zero exit code.
Here's my git diff :
diff --git a/meta-xfstk/recipes-xfstk/xfstk/xfstk_1.8.3.bb b/meta-xfstk/recipes-xfstk/xfstk/xfstk_1.8.3.bb
index ef57eaa..601bed9 100644
--- a/meta-xfstk/recipes-xfstk/xfstk/xfstk_1.8.3.bb
+++ b/meta-xfstk/recipes-xfstk/xfstk/xfstk_1.8.3.bb
@@ -6,7 +6,7 @@ inherit cmake qmake5_paths
BBCLASSEXTEND = "native"
-DEPENDS = "libxml2 libusb boost qtbase doxygen"
+DEPENDS = "libxml2-native libusb-native boost-native qtbase-native doxygen-native"
SRC_URI = "git://github.com/Exoyds/xfstk.git;tag=1.8.3;protocol=https"
Could you try a git diff? Maybe there's some small change somewhere?
I rebased first your branch on my (htoth/meta-intel-edison) btrfs branch (to save compile time). Then rebased my patch on top of that (the commit message now makes no sense, need to reword that later).
-DEPENDS = "libxml2 libusb boost qtbase doxygen"
+DEPENDS = "libxml2-native libusb-native boost-native qtbase-native"
I see that I manually checked out (didn't run setup) meta-qt5 zeus vs your jansa/zeus.
All in all no big differences.
Have you tried bitbake -c cleansstate xfstk-native
?
I have now checked bitbake -c cleansstate xfstk-native
, but it still fails at 86% compilation.
The good news is that it is 86% done, but I still cannot see why it cannot find libboost program_options. I think this is the last hurdle and we should be good after this.
Does your do_compile fail before or after 86%?
Also the error on my side looks to be an issue with libboost not being in the path, becuase I can see it builds and the file itself is present, just not linked to xfstk.
/factory/./platforms/baytrail/baytrailoptions.h:23:10: fatal error: boost/program_options.hpp: No such file or directory
| 23 | #include <boost/program_options.hpp>
| | ^~~~~~~~~~~~~~~~~~~~~~~~~~~
| compilation terminated.
vagrant@ubuntu2004:~/test/out/linux64/build$ locate program_options.hpp
/home/vagrant/test/out/linux64/build/tmp/sysroots-components/x86_64/boost-native/usr/include/boost/program_options.hpp
/home/vagrant/test/out/linux64/build/tmp/work/x86_64-linux/boost-native/1.71.0-r0/sysroot-destdir/home/vagrant/test/out/linux64/build/tmp/work/x86_64-linux/boost-native/1.71.0-r0/recipe-sysroot-native/usr/include/boost/program_options.hpp
/home/vagrant/test/out/linux64/build/tmp/work/x86_64-linux/xfstk-native/1.8.3-r0/git/external-dependencies/min_boost_1_46_1/boost/program_options.hpp
/home/vagrant/test/out/linux64/build/tmp/work/x86_64-linux/xfstk-native/1.8.3-r0/recipe-sysroot-native/usr/include/boost/program_options.hpp
It's hard to say. I just clean qtbase-native, recompiling that took between 4 - 5 min. Then it starts compiling xfstk and it fails almost immediately. It used this command line:
NOTE: VERBOSE=1 cmake --build /home/ferry/tmp/edison-intel/my/edison-morty/out/linux64/build/tmp/work/x86_64-linux/xfstk-native/1.8.3-r0/build --target xfstk-dldr-solo -- -j 8
So I went into out/linux64/build/tmp/work/x86_64-linux/xfstk-native/1.8.3-r0/temp/
and got log.do_compile
log.do_compile.txt. We might be able to compare those.
To me it seems there is something wrong with the include paths. recipe-sysroot-native/usr/include/QtCore
is included, but recipe-sysroot-native/usr/include
isn't
I couldn't find the xfstkdownloadermain.cpp entry that is mentioned in your log as the failed compilations. It also looks like we have different number of tasks as mine fails at 48/51 and yours at 3/4.
It does look to me as a path issue. I have tried adding PACKAGECONFIG_append_pn-boost = "program_options"
to my layer.conf to see if that would help, but nothing has changed.
For reference here are my logs:
So that is a very different log from mine.
So either configuration is different or my -j 8 is causing differences from your -j 2?
I have tried to set the -j to 8, but it still failed at the same point, where it couldn't find boost program options.
I have also checked and the -j is probed by bitbake when you run it. It sets it to the optimal value based on your processor, and so I guess that is why my was different - I run the compilation in a VM, so it has few cores.
Also I tried to capture this as well as I could - this is more or less where my compilation fails:
0: xfstk-native-1.8.3-r0 do_compile (pid 1514) 86%
It's easier to capture after doing bitbake -c cleansstate xfstk-native
as it goes slower then.
I've tried to check if it could be a CMake issue in xfstk itself, but so far that didn't prove to be the case.
AFAIKT CMake is not picking up the root include path for QtCore (in my case the bitbake variable ${STAGING_INCDIR_NATIVE}). And I don't know enough on CMake to patch that.
AFAIKT CMake is not picking up the root include path for QtCore (in my case the bitbake variable ${STAGING_INCDIR_NATIVE}). And I don't know enough on CMake to patch that.
If it is in the core or gui component I fixed some of the CMake flags there https://github.com/Exoyds/xfstk/commit/af9a18917ebeb0f7249e94bbcc98b655cc96b1fa. You could try the master of xfstk, it might work, but I have a feeling that it might be bitbake putting the libs in weird places.
I already found it and it now builds and installs without complaining. Though I think the install is not correct (but I have no idea when it would be correct, we might need to build it as sdk tool). My patches here 0b6eb139
But at least out/linux64/build/tmp/work/x86_64-linux/xfstk-native/1.8.3-r0/image/usr/bin/xfstk-dldr-solo -r
works.
That's great news! If it builds it's very close to working now.
I have pushed the changes and also created the 1.8.4 tag with the CMake changes.
I will re-deploy my VM and see if I can get it to build. If not I'll try on the host OS as it might be that bitbake is not a fan of being run in a VM.
I will re-deploy my VM and see if I can get it to build. If not I'll try on the host OS as it might be that bitbake is not a fan of being run in a VM.
It's just slow.
Okay, I've now compiled and installed successfully on a new VM. the only difference is that I still need to have doxygen as a dependency. Do you not get any errors on configure?
Bitbake configure log shows:
**** Begin Generate documenation ****
-- Found Doxygen: /usr/bin/doxygen (found version "1.8.17") found components: doxygen dot
I guess it must be detecting host doxygen and dot.
I found this example of a host tool (bmaptool): https://www.yoctoproject.org/docs/2.3/dev-manual/dev-manual.html#flashing-images-using-bmaptool.
So as I get it after we get xfstk built correctly we should be able to run it using oe-run-native xfstk-dldr-solo
Instead of trying to include doxygen and dot (dot is supplied by graphviz which pulls a lot of other stuff) I just removed generating the docs, see the 0f8c5c90dd9a9bb9ba0665a96f9c17a0c10de7e9
Also removed the -native suffixes as bmaptool does.
Still the packages get installed under out/linux64/build/tmp/work/x86_64-linux/xfstk-native/1.8.3-r0/image/
instead of out/linux64/build/tmp/work/x86_64-linux/xfstk-native/1.8.3-r0/recipe-sysroot-native/
. Maybe some non-standard installation process/options?
As a result oe-run-native xfstk-native xfstk-dldr-solo -r
can't find them.
I tried adding -caddto_recipe_sysroot
as the error message suggests, but without any result.
Instead of trying to include doxygen and dot (dot is supplied by graphviz which pulls a lot of other stuff) I just removed generating the docs, see the 0f8c5c9
I've pushed your patches to the branch. They work great!
Also removed the -native suffixes as bmaptool does.
Still the packages get installed under
out/linux64/build/tmp/work/x86_64-linux/xfstk-native/1.8.3-r0/image/
instead ofout/linux64/build/tmp/work/x86_64-linux/xfstk-native/1.8.3-r0/recipe-sysroot-native/
. Maybe some non-standard installation process/options?As a result
oe-run-native xfstk-native xfstk-dldr-solo -r
can't find them.I tried adding
-caddto_recipe_sysroot
as the error message suggests, but without any result.
I have ran into the same issue. So far I think that it either requires a do_install in the recipe, or maybe there are some paths hard-coded into the cmake files.
I have ran into the same issue. So far I think that it either requires a do_install in the recipe, or maybe there are some paths hard-coded into the cmake files.
Yeah, or we need some other magical bitbake variable. I think the problem happens during do_populate_sysroot.
I added a 2nd patch but am not sure if it is correct, haven't pushed it yet. But it does:
--- a/ancillary/configure/api/downloader-api/CMakeLists.txt
+++ b/ancillary/configure/api/downloader-api/CMakeLists.txt
@@ -70,7 +70,7 @@ SET_TARGET_PROPERTIES(xfstk-dldr-api
INSTALL(TARGETS xfstk-dldr-gui-api
LIBRARY
- DESTINATION lib${LIBSUFFIX}
+ DESTINATION /lib${LIBSUFFIX}
NAMELINK_SKIP
COMPONENT lib)
What do you think, should there be a /
? I'm new to cmake ...
With the patch it installs libxfstk-dldr-gui-api.so
to image/lib/
. Although when building out of Yocto it goes to /usr/local/lib/
.
It doesn't fix copying the files to recipe-sysroot-native
yet, unfortunately.
From what I could see, it might not even be cmake's fault.
Looks like ${D} is set to ${WORKDIR}/image even for a native recipe. ( https://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#var-D ) . So I think this may be the issue. I'm trying to find if there is a solution out there, other than overwriting D to something else.
Under tmp I found run.sysroot_stage_all. From this I get my last patch is exactly wrong. It seems stuff needs to be under image/home/user/long-path-to/recipe-sysroot-native and then gets staged to sysroot-destdir.
I tried to do something like this:
addtask copy_files_sysroot after do_populate_sysroot before do_package
do_copy_files_sysroot() {
cp --no-preserve=ownership ${D}${bindir}/xfstk-dldr-solo ${SYSROOT_DESTDIR}${bindir}/xfstk-dldr-solo
}
but this still does not seem to work. Looks like ${D}${bindir}/xfstk-dldr-solo
is not quite correct. This may be why it does not get staged in do_populate_sysroot
I believe cmake install under the wrong directory (/image/usr/bin/xfstk-dldr-solo but should be /image/home/user/long-path-to/recipe-sysroot-native/usr/bin/xfstk-dldr-solo) but did not trace that yet.
The only file that gets installed in the right place is libxfstk-dldr-gui-api.so
.
I tried to do this: https://github.com/Exoyds/xfstk/commit/56272920cf3fd92905e063bc2b47e9e66da58900 to force xfstk to use GNU paths, but it made no change at all.
The binary is still in /home/vagrant/test/out/linux64/build/tmp/work/x86_64-linux/xfstk-native/1.8.4-r0/image/usr/bin/xfstk-dldr-solo
I commented on your patch here.
I got the idea from log.do_configure
:
CMake Warning:
Manually-specified variables were not used by the project:
CMAKE_INSTALL_BINDIR
CMAKE_INSTALL_DATAROOTDIR
CMAKE_INSTALL_INCLUDEDIR
CMAKE_INSTALL_LIBDIR
CMAKE_INSTALL_LIBEXECDIR
CMAKE_INSTALL_LOCALSTATEDIR
CMAKE_INSTALL_SBINDIR
CMAKE_INSTALL_SHAREDSTATEDIR
CMAKE_INSTALL_SYSCONFDIR
WARNING_AS_ERROR
2 questions arise: 1) Do we need to fix all CMakeLists.txt destinations? 2) Do we do that as a patch applied by bitbake, or do we patch in upstream (your) git? Will building outside of Yocto still result in a correct deb?
BTW cmake.bbclass binds these variables to bitbake variables here.
I commented on your patch here.
Yup, just seen that, great news!
2 questions arise:
1. Do we need to fix all CMakeLists.txt destinations?
I would do, just in case. In fact I'll get on it now, and probably get it done tonight.
- Do we do that as a patch applied by bitbake, or do we patch in upstream (your) git? Will building outside of Yocto still result in a correct deb?
- I will patch it today, and see if it builds. If it still builds fine outside of yocto I'll push that upstream.
BTW cmake.bbclass binds these variables to bitbake variables here.
This is good, as it means I can patch everything, and it should still be correct as per CMake itself
Okay, so now with bitbake xfstk-native -caddto_recipe_sysroot
it builds and adds to the sysroot and then we just run oe-run-native xfstk-native xfstk-dldr-solo -r
and it all seems to work!
Thank you so much! It was a pleasure working with you! I would say at least half of this pull request is yours, and I am very grateful for your help! :)
I have made all the changes in this branch so please let me know if this is okay, or if I squashed too much. If all is good I will pull request that branch and leave the other one as an archive on my fork.
I have also tested the rebase and any changes, just in case, and they all still work!
I merged your [branch](https://github.com/Exoyds/meta-intel-edison/tree/xfstk-recipe-rebase, thanks!
Resolves #8, when finished.