genodelabs / genode-world

Collection of community-maintained components for Genode
Other
45 stars 47 forks source link

qt5_webengine: compile requires Python 2 (which has been sunset in 2020) #340

Open mewmew opened 1 year ago

mewmew commented 1 year ago

I wanted to try and compile Falkon, and as a prerequisite tried to compile the qt5_webengine library from genode-world staging branch (ref 9e06bf73cc5313b0530e68568c8fd516817b0bcf).

Command used to build qt5_webengine:

./tool/depot/create FORCE=1 REBUILD= UPDATE_VERSIONS=1 mewmew/bin/x86_64/qt5_webengine

However, the following build error was presented:

Project WARNING: Cross compiling without sysroot. Disabling pkg-config.
sh: line 1: --exists: command not found

Note: I am able to build qt5_base successfully (using ./tool/depot/create FORCE=1 REBUILD= UPDATE_VERSIONS=1 mewmew/bin/x86_64/qt5_base). The same is true for each of qt5_component, qt5_declarative, qt5_quickcontrols, qt5_svg and qt5_webchannel.

Below follows the full build output for qt5_webengine which gave the above error message.

detailed build output ``` $ ./tool/depot/create FORCE=1 REBUILD= UPDATE_VERSIONS=1 mewmew/bin/x86_64/qt5_webengine created mewmew/api/expat/2022-11-16 created mewmew/api/libc/2023-05-26 created mewmew/api/mesa/2021-08-28 created mewmew/api/nss3/2023-04-27 created mewmew/api/qt5_base/2023-05-26 created mewmew/api/qt5_declarative/2023-05-26 created mewmew/api/qt5_webchannel/2023-05-26 created mewmew/api/qt5_webengine/2023-05-26 created mewmew/api/so/2020-05-17 created mewmew/api/stdcxx/2023-06-15 created mewmew/src/qt5_webengine/2023-09-12 checking library dependencies... Library ldso_so_support COMPILE so_support.o MERGE ldso_so_support.lib.a Library expat CONVERT expat.symbols.s ASSEMBLE expat.symbols.o MERGE expat.abi.so Library libc CONVERT libc.symbols.s ASSEMBLE libc.symbols.o MERGE libc.abi.so Library mesa CONVERT mesa.symbols.s ASSEMBLE mesa.symbols.o MERGE mesa.abi.so Library nss3 CONVERT nss3.symbols.s ASSEMBLE nss3.symbols.o MERGE nss3.abi.so Library qt5_base MERGE qt5_base.lib.a Library qt5_declarative MERGE qt5_declarative.lib.a Library qt5_webchannel MERGE qt5_webchannel.lib.a Library egl CONVERT egl.symbols.s ASSEMBLE egl.symbols.o MERGE egl.abi.so Library libm CONVERT libm.symbols.s ASSEMBLE libm.symbols.o MERGE libm.abi.so Library stdcxx CONVERT stdcxx.symbols.s ASSEMBLE stdcxx.symbols.o MERGE stdcxx.abi.so Library libQt5Core CONVERT libQt5Core.symbols.s ASSEMBLE libQt5Core.symbols.o MERGE libQt5Core.abi.so Library libQt5Gui CONVERT libQt5Gui.symbols.s ASSEMBLE libQt5Gui.symbols.o MERGE libQt5Gui.abi.so Library libQt5Network CONVERT libQt5Network.symbols.s ASSEMBLE libQt5Network.symbols.o MERGE libQt5Network.abi.so Library libQt5Widgets CONVERT libQt5Widgets.symbols.s ASSEMBLE libQt5Widgets.symbols.o MERGE libQt5Widgets.abi.so Library libQt5Qml CONVERT libQt5Qml.symbols.s ASSEMBLE libQt5Qml.symbols.o MERGE libQt5Qml.abi.so Library libQt5QmlModels CONVERT libQt5QmlModels.symbols.s ASSEMBLE libQt5QmlModels.symbols.o MERGE libQt5QmlModels.abi.so Library libQt5Quick CONVERT libQt5Quick.symbols.s ASSEMBLE libQt5Quick.symbols.o MERGE libQt5Quick.abi.so Library libQt5QuickWidgets CONVERT libQt5QuickWidgets.symbols.s ASSEMBLE libQt5QuickWidgets.symbols.o MERGE libQt5QuickWidgets.abi.so Library libQt5WebChannel CONVERT libQt5WebChannel.symbols.s ASSEMBLE libQt5WebChannel.symbols.o MERGE libQt5WebChannel.abi.so Running configuration tests... Checking for architecture supported... yes Checking for bison... yes Checking for flex... yes Checking for gperf... yes Checking for host pkg-config... /usr/bin/pkg-config Checking for jumbo build merge limit... 8 Checking for platform supported... Checking for linker supports -z noexecstack... no Checking for build path without whitespace... yes Checking for python2... Checking for submodule initialized... yes Checking for d-bus... no Checking for fontconfig... no Checking for glib-2.0 >= 2.32.0... no Checking for jsoncpp... no Checking for khr... no Checking for lcms2... no Checking for libdrm... no Checking for libevent... no Checking for libvpx... no Checking for compatible libxml2 and libxslt... no Checking for minizip... no Checking for nss >= 3.26... no Checking for protobuf... no Checking for re2... no Checking for snappy... no Checking for x11... no Checking for xcomposite... no Checking for xcursor... no Checking for xi... no Checking for xtst... no Checking for node.js... yes Checking for poppler-cpp... no Checking for host compiler... yes Done running configuration tests. Configure summary: Qt WebEngine Build Tools: Use System Ninja ....................... no Use System Gn .......................... no Jumbo Build Merge Limit ................ 8 Developer build ........................ no Sanitizer .............................. no QtWebEngine required system libraries: fontconfig ........................... no dbus ................................. no nss .................................. no khr .................................. no glibc ................................ no Optional system libraries used: re2 .................................. no icu .................................. no libwebp, libwebpmux and libwebpdemux . no opus ................................. no ffmpeg ............................... no libvpx ............................... no snappy ............................... no glib ................................. no zlib ................................. no minizip .............................. no libevent ............................. no jsoncpp .............................. no protobuf ............................. no libxml2 and libxslt .................. no lcms2 ................................ no png .................................. no JPEG ................................. no harfbuzz ............................. no freetype ............................. no Qt WebEngineCore: Embedded build ......................... no Full debug information ................. no Pepper Plugins ......................... no Printing and PDF ....................... no Proprietary Codecs ..................... yes Spellchecker ........................... no Native Spellchecker .................... no WebRTC ................................. yes PipeWire over GIO ...................... no Geolocation ............................ no WebChannel support ..................... yes Kerberos Authentication ................ no Extensions ............................. no Node.js ................................ yes Support qpa-xcb ........................ no Building v8 snapshot supported ......... yes Use ALSA ............................... no Use PulseAudio ......................... no Use OSS ................................ yes Use sndio .............................. no Qt WebEngineQml: Support Qt WebEngine Qml ............... yes UI Delegates ........................... yes Test Support ........................... no Qt WebEngineWidgets: Support Qt WebEngine Widgets ........... yes Note: The following modules are not being compiled in this configuration: pdf pdfwidgets WARNING: Python version 2 (2.7.5 or later) is required to build QtPdf. WARNING: QtPdf will not be built. Qt is now configured for building. Just run 'make'. Once everything is built, you must run 'make install'. Qt will be installed into '/home/u/Desktop/sculpt_play/genode/depot/mewmew/bin/x86_64/qt5_webengine/2023-09-12.build/var/libcache/qt5_webengine/qmake_root'. Prior to reconfiguration, make sure you remove any leftovers from the previous build. /home/u/Desktop/sculpt_play/genode/depot/mewmew/src/qt5_webengine/2023-09-12/src/lib/qt5/qtwebengine/src/3rdparty/ninja/configure.py:26: DeprecationWarning: 'pipes' is deprecated and slated for removal in Python 3.13 import pipes /home/u/Desktop/sculpt_play/genode/depot/mewmew/src/qt5_webengine/2023-09-12/src/lib/qt5/qtwebengine/src/3rdparty/ninja/src/build_log.cc: In function ‘uint64_t {anonymous}::MurmurHash64A(const void*, size_t)’: /home/u/Desktop/sculpt_play/genode/depot/mewmew/src/qt5_webengine/2023-09-12/src/lib/qt5/qtwebengine/src/3rdparty/ninja/src/build_log.cc:78:13: warning: this statement may fall through [-Wimplicit-fallthrough=] 78 | case 7: h ^= uint64_t(data[6]) << 48; | ~~^~~~~~~~~~~~~~~~~~~~~~~~~~ /home/u/Desktop/sculpt_play/genode/depot/mewmew/src/qt5_webengine/2023-09-12/src/lib/qt5/qtwebengine/src/3rdparty/ninja/src/build_log.cc:79:3: note: here 79 | case 6: h ^= uint64_t(data[5]) << 40; | ^~~~ /home/u/Desktop/sculpt_play/genode/depot/mewmew/src/qt5_webengine/2023-09-12/src/lib/qt5/qtwebengine/src/3rdparty/ninja/src/build_log.cc:79:13: warning: this statement may fall through [-Wimplicit-fallthrough=] 79 | case 6: h ^= uint64_t(data[5]) << 40; | ~~^~~~~~~~~~~~~~~~~~~~~~~~~~ /home/u/Desktop/sculpt_play/genode/depot/mewmew/src/qt5_webengine/2023-09-12/src/lib/qt5/qtwebengine/src/3rdparty/ninja/src/build_log.cc:80:3: note: here 80 | case 5: h ^= uint64_t(data[4]) << 32; | ^~~~ /home/u/Desktop/sculpt_play/genode/depot/mewmew/src/qt5_webengine/2023-09-12/src/lib/qt5/qtwebengine/src/3rdparty/ninja/src/build_log.cc:80:13: warning: this statement may fall through [-Wimplicit-fallthrough=] 80 | case 5: h ^= uint64_t(data[4]) << 32; | ~~^~~~~~~~~~~~~~~~~~~~~~~~~~ /home/u/Desktop/sculpt_play/genode/depot/mewmew/src/qt5_webengine/2023-09-12/src/lib/qt5/qtwebengine/src/3rdparty/ninja/src/build_log.cc:81:3: note: here 81 | case 4: h ^= uint64_t(data[3]) << 24; | ^~~~ /home/u/Desktop/sculpt_play/genode/depot/mewmew/src/qt5_webengine/2023-09-12/src/lib/qt5/qtwebengine/src/3rdparty/ninja/src/build_log.cc:81:13: warning: this statement may fall through [-Wimplicit-fallthrough=] 81 | case 4: h ^= uint64_t(data[3]) << 24; | ~~^~~~~~~~~~~~~~~~~~~~~~~~~~ /home/u/Desktop/sculpt_play/genode/depot/mewmew/src/qt5_webengine/2023-09-12/src/lib/qt5/qtwebengine/src/3rdparty/ninja/src/build_log.cc:82:3: note: here 82 | case 3: h ^= uint64_t(data[2]) << 16; | ^~~~ /home/u/Desktop/sculpt_play/genode/depot/mewmew/src/qt5_webengine/2023-09-12/src/lib/qt5/qtwebengine/src/3rdparty/ninja/src/build_log.cc:82:13: warning: this statement may fall through [-Wimplicit-fallthrough=] 82 | case 3: h ^= uint64_t(data[2]) << 16; | ~~^~~~~~~~~~~~~~~~~~~~~~~~~~ /home/u/Desktop/sculpt_play/genode/depot/mewmew/src/qt5_webengine/2023-09-12/src/lib/qt5/qtwebengine/src/3rdparty/ninja/src/build_log.cc:83:3: note: here 83 | case 2: h ^= uint64_t(data[1]) << 8; | ^~~~ /home/u/Desktop/sculpt_play/genode/depot/mewmew/src/qt5_webengine/2023-09-12/src/lib/qt5/qtwebengine/src/3rdparty/ninja/src/build_log.cc:83:13: warning: this statement may fall through [-Wimplicit-fallthrough=] 83 | case 2: h ^= uint64_t(data[1]) << 8; | ~~^~~~~~~~~~~~~~~~~~~~~~~~~ /home/u/Desktop/sculpt_play/genode/depot/mewmew/src/qt5_webengine/2023-09-12/src/lib/qt5/qtwebengine/src/3rdparty/ninja/src/build_log.cc:84:3: note: here 84 | case 1: h ^= uint64_t(data[0]); | ^~~~ In file included from /home/u/Desktop/sculpt_play/genode/depot/mewmew/src/qt5_webengine/2023-09-12/src/lib/qt5/qtwebengine/src/3rdparty/ninja/src/build_log.h:22, from /home/u/Desktop/sculpt_play/genode/depot/mewmew/src/qt5_webengine/2023-09-12/src/lib/qt5/qtwebengine/src/3rdparty/ninja/src/build_log.cc:23: /home/u/Desktop/sculpt_play/genode/depot/mewmew/src/qt5_webengine/2023-09-12/src/lib/qt5/qtwebengine/src/3rdparty/ninja/src/hash_map.h: In function ‘unsigned int MurmurHash2(const void*, size_t)’: /home/u/Desktop/sculpt_play/genode/depot/mewmew/src/qt5_webengine/2023-09-12/src/lib/qt5/qtwebengine/src/3rdparty/ninja/src/hash_map.h:42:13: warning: this statement may fall through [-Wimplicit-fallthrough=] 42 | case 3: h ^= data[2] << 16; | ~~^~~~~~~~~~~~~~~~ /home/u/Desktop/sculpt_play/genode/depot/mewmew/src/qt5_webengine/2023-09-12/src/lib/qt5/qtwebengine/src/3rdparty/ninja/src/hash_map.h:43:3: note: here 43 | case 2: h ^= data[1] << 8; | ^~~~ /home/u/Desktop/sculpt_play/genode/depot/mewmew/src/qt5_webengine/2023-09-12/src/lib/qt5/qtwebengine/src/3rdparty/ninja/src/hash_map.h:43:13: warning: this statement may fall through [-Wimplicit-fallthrough=] 43 | case 2: h ^= data[1] << 8; | ~~^~~~~~~~~~~~~~~ /home/u/Desktop/sculpt_play/genode/depot/mewmew/src/qt5_webengine/2023-09-12/src/lib/qt5/qtwebengine/src/3rdparty/ninja/src/hash_map.h:44:3: note: here 44 | case 1: h ^= data[0]; | ^~~~ In file included from /home/u/Desktop/sculpt_play/genode/depot/mewmew/src/qt5_webengine/2023-09-12/src/lib/qt5/qtwebengine/src/3rdparty/ninja/src/state.h:25, from /home/u/Desktop/sculpt_play/genode/depot/mewmew/src/qt5_webengine/2023-09-12/src/lib/qt5/qtwebengine/src/3rdparty/ninja/src/state.cc:15: /home/u/Desktop/sculpt_play/genode/depot/mewmew/src/qt5_webengine/2023-09-12/src/lib/qt5/qtwebengine/src/3rdparty/ninja/src/hash_map.h: In function ‘unsigned int MurmurHash2(const void*, size_t)’: /home/u/Desktop/sculpt_play/genode/depot/mewmew/src/qt5_webengine/2023-09-12/src/lib/qt5/qtwebengine/src/3rdparty/ninja/src/hash_map.h:42:13: warning: this statement may fall through [-Wimplicit-fallthrough=] 42 | case 3: h ^= data[2] << 16; | ~~^~~~~~~~~~~~~~~~ /home/u/Desktop/sculpt_play/genode/depot/mewmew/src/qt5_webengine/2023-09-12/src/lib/qt5/qtwebengine/src/3rdparty/ninja/src/hash_map.h:43:3: note: here 43 | case 2: h ^= data[1] << 8; | ^~~~ /home/u/Desktop/sculpt_play/genode/depot/mewmew/src/qt5_webengine/2023-09-12/src/lib/qt5/qtwebengine/src/3rdparty/ninja/src/hash_map.h:43:13: warning: this statement may fall through [-Wimplicit-fallthrough=] 43 | case 2: h ^= data[1] << 8; | ~~^~~~~~~~~~~~~~~ /home/u/Desktop/sculpt_play/genode/depot/mewmew/src/qt5_webengine/2023-09-12/src/lib/qt5/qtwebengine/src/3rdparty/ninja/src/hash_map.h:44:3: note: here 44 | case 1: h ^= data[0]; | ^~~~ Project MESSAGE: /home/u/Desktop/sculpt_play/genode/depot/mewmew/src/qt5_webengine/2023-09-12/src/lib/qt5/qtwebengine/src/3rdparty/gn/build/gen.py --no-last-commit-position --out-path /home/u/Desktop/sculpt_play/genode/depot/mewmew/bin/x86_64/qt5_webengine/2023-09-12.build/var/libcache/qt5_webengine/src/3rdparty/gn/out/Release --cc "/usr/bin/gcc" --cxx "/usr/bin/g++" --ld "/usr/bin/g++" Project WARNING: Cross compiling without sysroot. Disabling pkg-config. sh: line 1: --exists: command not found Project MESSAGE: Running: /home/u/Desktop/sculpt_play/genode/depot/mewmew/bin/x86_64/qt5_webengine/2023-09-12.build/var/libcache/qt5_webengine/src/3rdparty/gn/out/Release/gn gen /home/u/Desktop/sculpt_play/genode/depot/mewmew/bin/x86_64/qt5_webengine/2023-09-12.build/var/libcache/qt5_webengine/src/core/release --script-executable=python --args='use_qt=true init_stack_vars=false is_component_build=false is_shared=true enable_debugallocation=false enable_media_remoting=false enable_message_center=false enable_nacl=false enable_remoting=false enable_reporting=false enable_resource_whitelist_generation=false enable_swiftshader=false angle_enable_swiftshader=false enable_web_auth=true enable_web_speech=false enable_widevine=true forbid_non_component_debug_builds=false has_native_accessibility=false safe_browsing_mode=0 toolkit_views=false treat_warnings_as_errors=false use_allocator_shim=false use_allocator="none" use_custom_libcxx=false closure_compile=false optimize_webui=false v8_use_external_startup_data=false strip_absolute_paths_from_debug_symbols=false use_jumbo_build=true jumbo_file_merge_limit=8 jumbo_build_excluded=["browser"] enable_precompiled_headers=false is_official_build=true is_debug=false symbol_level=1 blink_symbol_level=0 remove_v8base_debug_symbols=true use_cups=false use_gio=false use_gnome_keyring=false linux_use_bundled_binutils=false use_udev=true use_bundled_fontconfig=true use_sysroot=false enable_session_service=false is_cfi=false use_ozone=true ozone_auto_platforms=false ozone_platform_headless=false ozone_platform_external=true ozone_platform="qt" ozone_extra_path="/home/u/Desktop/sculpt_play/genode/depot/mewmew/src/qt5_webengine/2023-09-12/src/lib/qt5/qtwebengine/src/core/ozone/ozone_extra.gni" use_gold=false use_lld=false is_clang=false custom_toolchain="/home/u/Desktop/sculpt_play/genode/depot/mewmew/bin/x86_64/qt5_webengine/2023-09-12.build/var/libcache/qt5_webengine/src/toolchain:target" host_toolchain="/home/u/Desktop/sculpt_play/genode/depot/mewmew/bin/x86_64/qt5_webengine/2023-09-12.build/var/libcache/qt5_webengine/src/toolchain:host" v8_snapshot_toolchain="/home/u/Desktop/sculpt_play/genode/depot/mewmew/bin/x86_64/qt5_webengine/2023-09-12.build/var/libcache/qt5_webengine/src/toolchain:v8_snapshot" target_cpu="x64" target_os="freebsd" use_system_libjpeg=false use_system_freetype=false use_system_harfbuzz=false use_glib=false enable_basic_printing=false enable_print_preview=false use_dbus=false use_udev=false enable_basic_printing=false enable_print_preview=false enable_pdf=false enable_plugins=false enable_spellcheck=false enable_webrtc=true proprietary_codecs=true ffmpeg_branding="Chrome" rtc_use_h264=true enable_extensions=false use_kerberos=false have_nodejs=true use_pulseaudio=false use_alsa=false use_sndio=false use_oss=true use_libpci=false qtwebengine_target="/home/u/Desktop/sculpt_play/genode/depot/mewmew/bin/x86_64/qt5_webengine/2023-09-12.build/var/libcache/qt5_webengine/src/core/release:QtWebEngineCore"' --root=/home/u/Desktop/sculpt_play/genode/depot/mewmew/src/qt5_webengine/2023-09-12/src/lib/qt5/qtwebengine/src/3rdparty/chromium make[10]: *** [Makefile.gn_run:346: run_ninja] Error 1 make[9]: *** [Makefile:82: sub-gn_run-pro-make_first] Error 2 make[8]: *** [Makefile:77: sub-core-make_first] Error 2 make[7]: *** [Makefile:46: sub-src] Error 2 make[6]: *** [/home/u/Desktop/sculpt_play/genode/depot/mewmew/src/qt5_webengine/2023-09-12/lib/mk/qt5_webengine.inc:31: built.tag] Error 2 make[5]: *** [var/libdeps:402: qt5_webengine.lib] Error 2 make[4]: *** [Makefile:336: gen_deps_and_build_targets] Error 2 make[3]: *** [/home/u/Desktop/sculpt_play/genode/tool/depot/mk/build_bin_archive:208: /home/u/Desktop/sculpt_play/genode/depot/mewmew/bin/x86_64/qt5_webengine/2023-09-12.build/bin] Error 1 make[2]: *** [/home/u/Desktop/sculpt_play/genode/depot/var/build.mk:12: mewmew/bin/x86_64/qt5_webengine/2023-09-12] Error 2 make[1]: *** [/home/u/Desktop/sculpt_play/genode/tool/depot/build:136: execute_generated_build_mk_file] Error 2 make: *** [tool/depot/create:59: build] Error 2 ```

Edit: the "Cross compiling without sysroot. Disabling pkg-config" message seems to be from configure templates from qt5_base:

grep for Cross compiling without sysroot in depot/mewmew/src/qt5_webengine:

# qt5_base/2023-09-12/src/lib/qt5/qtbase/configure.pri:
qtConfAddWarning("Cross compiling without sysroot. Disabling pkg-config")
# qt5_base/2023-09-12/src/lib/qt5/qtbase/mkspecs/features/qt_functions.prf:
warning("Cross compiling without sysroot. Disabling pkg-config.")

And the "--exists command not found" error is likely related to the $PKG_CONFIG variable not being set.

grep for --exists in depot/mewmew/src/qt5_webengine:

# src/lib/qt5/qtwebengine/src/3rdparty/chromium/third_party/fontconfig/src/m4/pkg.m4:
AC_RUN_LOG([$PKG_CONFIG --exists --print-errors "$1"]); then
# src/lib/qt5/qtwebengine/src/3rdparty/chromium/third_party/ffmpeg/configure:
test_cmd $pkg_config --exists --print-errors $pkg_version || return
# src/lib/qt5/qtwebengine/src/3rdparty/chromium/third_party/libxml/src/aclocal.m4:
AC_RUN_LOG([$PKG_CONFIG --exists --print-errors "$1"]); then
# src/lib/qt5/qtwebengine/src/3rdparty/chromium/third_party/freetype/src/builds/unix/pkg.m4:
AC_RUN_LOG([$PKG_CONFIG --exists --print-errors "$1"]); then
# src/lib/qt5/qtwebengine/src/3rdparty/chromium/third_party/icu/source/aclocal.m4:
AC_RUN_LOG([$PKG_CONFIG --exists --print-errors "$1"]); then
# src/lib/qt5/qtwebengine/src/3rdparty/chromium/third_party/icu/source/configure:
{ { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"icu-le-hb\""; } >&5
# src/lib/qt5/qtwebengine/src/3rdparty/chromium/third_party/icu/source/configure:
($PKG_CONFIG --exists --print-errors "icu-le-hb") 2>&5
# src/lib/qt5/qtwebengine/src/3rdparty/chromium/third_party/icu/source/configure:
{ { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"icu-le-hb\""; } >&5
# src/lib/qt5/qtwebengine/src/3rdparty/chromium/third_party/icu/source/configure:
($PKG_CONFIG --exists --print-errors "icu-le-hb") 2>&5
cproc commented 1 year ago

I get the --exists: command not found message too, but the actual problem is probably something different. One common error when building qt5_webengine is memory exhaustion and the Linux kernel killing GCC. Do you see a log message about that with dmesg? In that case, you could reduce the number of parallel ninja jobs with NINJAJOBS=-j... and calculate with 2G of RAM per job when choosing the number (https://github.com/genodelabs/genode-world/issues/305). You can also add VERBOSE= to the command line to see more log messages during the build process.

mewmew commented 1 year ago

Thanks a lot @cproc! That helped a lot to pinpoint the problem.

Just for debugging purposes, I set -j1 (to prevent concurrent log output), and enabled VERBOSE=

Now it fails due to a python error (cannot import name 'Mapping' from 'collections'):

$ time ./tool/depot/create NINJAJOBS=-j1 VERBOSE= FORCE=1 REBUILD= UPDATE_VERSIONS=1 mewmew/bin/x86_64/qt5_webengine

...

[142/21294] touch obj/cc/mojom/mojom__type_mappings__validator.stamp
[143/21294] touch obj/cc/base/base__jumbo_merge.stamp
[144/21294] touch obj/chrome/browser/chrome_internal_resources_gen.stamp
[145/21294] python ../../../../../../../../../../src/qt5_webengine/2023-09-12/src/lib/qt5/qtwebengine/src/3rdparty/chromium/build/write_buildflag_header.py --output chrome/common/buildflags.h --rulename //chrome/common:buildflags --gen-dir gen --definitions 7142641371257702246.rsp
[146/21294] python ../../../../../../../../../../src/qt5_webengine/2023-09-12/src/lib/qt5/qtwebengine/src/3rdparty/chromium/mojo/public/tools/bindings/validate_typemap_config.py //components/services/filesystem/public/mojom/mojom:mojom /home/u/Desktop/sculpt_play/genode/depot/mewmew/bin/x86_64/qt5_webengine/2023-09-12.build/var/libcache/qt5_webengine/src/core/release/gen/components/services/filesystem/public/mojom/mojom_blink.typemap_config /home/u/Desktop/sculpt_play/genode/depot/mewmew/bin/x86_64/qt5_webengine/2023-09-12.build/var/libcache/qt5_webengine/src/core/release/gen/components/services/filesystem/public/mojom/mojom_blink.typemap_config.validated
[147/21294] touch obj/chrome/common/buildflags.stamp
[148/21294] python ../../../../../../../../../../src/qt5_webengine/2023-09-12/src/lib/qt5/qtwebengine/src/3rdparty/chromium/mojo/public/tools/mojom/mojom_parser.py --input-root /home/u/Desktop/sculpt_play/genode/depot/mewmew/src/qt5_webengine/2023-09-12/src/lib/qt5/qtwebengine/src/3rdparty/chromium/ --input-root /home/u/Desktop/sculpt_play/genode/depot/mewmew/bin/x86_64/qt5_webengine/2023-09-12.build/var/libcache/qt5_webengine/src/core/release/gen --output-root /home/u/Desktop/sculpt_play/genode/depot/mewmew/bin/x86_64/qt5_webengine/2023-09-12.build/var/libcache/qt5_webengine/src/core/release/gen --mojom-file-list=12959865804793651387..rsp --enable-feature is_posix --enable-feature is_linux
[149/21294] touch obj/chrome/browser/resource_coordinator/mojo_bindings__parser.stamp
[150/21294] touch obj/components/apdu/apdu.inputdeps.stamp
[151/21294] /usr/local/genode/tool/23.05/bin/genode-x86-g++ -MMD -MF obj/components/apdu/apdu/apdu_command.o.d -DIS_APDU_IMPL -DUSE_AURA=1 -DUSE_NSS_CERTS=1 -DUSE_OZONE=1 -DOFFICIAL_BUILD -DTOOLKIT_QT -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -DNO_UNWIND_TABLES -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -DNDEBUG -DNVALGRIND -DDYNAMIC_ANNOTATIONS_ENABLED=0 -Igen -I../../../../../../../../../../src/qt5_webengine/2023-09-12/src/lib/qt5/qtwebengine/src/3rdparty/chromium -I../../../../../../../../../../src/qt5_webengine/2023-09-12/src/lib/qt5/qtwebengine/src/3rdparty/chromium/third_party/boringssl/src/include -fno-strict-aliasing --param=ssp-buffer-size=4 -fstack-protector -fno-unwind-tables -fno-asynchronous-unwind-tables -fPIC -pipe -m64 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -Wno-unused-local-typedefs -Wno-maybe-uninitialized -Wno-deprecated-declarations -fno-delete-null-pointer-checks -Wno-comments -Wno-packed-not-aligned -Wno-dangling-else -Wno-missing-field-initializers -Wno-unused-parameter -O2 -fno-ident -fdata-sections -ffunction-sections -fno-omit-frame-pointer -g1 -fvisibility=hidden -std=gnu++14 -Wno-narrowing -Wno-class-memaccess -Wno-attributes -Wno-class-memaccess -Wno-subobject-linkage -Wno-invalid-offsetof -Wno-return-type -Wno-deprecated-copy -fno-exceptions -fno-rtti -fvisibility-inlines-hidden -pipe -pipe -D__FreeBSD__=12 -D__GENODE__ -ffunction-sections -fno-strict-aliasing -nostdinc -m64 -mcmodel=large -fPIC -I/home/u/Desktop/sculpt_play/genode/depot/mewmew/api/mesa/2021-08-28/include/drm-uapi -I/home/u/Desktop/sculpt_play/genode/depot/mewmew/api/nss3/2023-04-27/include/nspr -I/home/u/Desktop/sculpt_play/genode/depot/mewmew/api/nss3/2023-04-27/include/nss -I/home/u/Desktop/sculpt_play/genode/depot/mewmew/api/stdcxx/2023-06-15/include/stdcxx -I/home/u/Desktop/sculpt_play/genode/depot/mewmew/api/stdcxx/2023-06-15/include/stdcxx/std -I/home/u/Desktop/sculpt_play/genode/depot/mewmew/api/stdcxx/2023-06-15/include/stdcxx/c_global -I/home/u/Desktop/sculpt_play/genode/depot/mewmew/api/stdcxx/2023-06-15/include/stdcxx/../spec/x86_64/stdcxx -I/home/u/Desktop/sculpt_play/genode/depot/mewmew/api/libc/2023-05-26/include/spec/x86 -I/home/u/Desktop/sculpt_play/genode/depot/mewmew/api/libc/2023-05-26/include/spec/x86_64 -I/home/u/Desktop/sculpt_play/genode/depot/mewmew/api/stdcxx/2023-06-15/include/spec/x86_64 -I/home/u/Desktop/sculpt_play/genode/depot/mewmew/api/libc/2023-05-26/include/libc -I/home/u/Desktop/sculpt_play/genode/depot/mewmew/api/libc/2023-05-26/include/spec/x86_64/libc -I/home/u/Desktop/sculpt_play/genode/depot/mewmew/api/libc/2023-05-26/include/spec/x86/libc -I/home/u/Desktop/sculpt_play/genode/depot/mewmew/api/libc/2023-05-26/include/libc-genode -I/home/u/Desktop/sculpt_play/genode/depot/mewmew/api/libc/2023-05-26/include/libc -I/home/u/Desktop/sculpt_play/genode/depot/mewmew/api/libc/2023-05-26/include/spec/x86_64/libc -I/home/u/Desktop/sculpt_play/genode/depot/mewmew/api/libc/2023-05-26/include/spec/x86/libc -I/home/u/Desktop/sculpt_play/genode/depot/mewmew/api/libc/2023-05-26/include/libc-genode -I/home/u/Desktop/sculpt_play/genode/depot/mewmew/api/expat/2022-11-16/include -I/home/u/Desktop/sculpt_play/genode/depot/mewmew/api/libc/2023-05-26/include -I/home/u/Desktop/sculpt_play/genode/depot/mewmew/api/mesa/2021-08-28/include -I/home/u/Desktop/sculpt_play/genode/depot/mewmew/api/nss3/2023-04-27/include -I/home/u/Desktop/sculpt_play/genode/depot/mewmew/api/qt5_base/2023-05-26/include -I/home/u/Desktop/sculpt_play/genode/depot/mewmew/api/stdcxx/2023-06-15/include -I/usr/local/genode/tool/23.05/bin/../lib/gcc/x86_64-pc-elf/12.3.0/include -I/home/u/Desktop/sculpt_play/genode/depot/mewmew/bin/x86_64/qt5_webengine/2023-09-12.build/var/libcache/qt5_webengine/qmake_root/include/QtCore/spec/genode-x86_64-g++ -fpermissive -c ../../../../../../../../../../src/qt5_webengine/2023-09-12/src/lib/qt5/qtwebengine/src/3rdparty/chromium/components/apdu/apdu_command.cc -o obj/components/apdu/apdu/apdu_command.o
[152/21294] python ../../../../../../../../../../src/qt5_webengine/2023-09-12/src/lib/qt5/qtwebengine/src/3rdparty/chromium/tools/polymer/polymer.py --js_file app.js --html_file app.html --html_type v3-ready --in_folder ../../../../../../../../../../src/qt5_webengine/2023-09-12/src/lib/qt5/qtwebengine/src/3rdparty/chromium/chrome/browser/resources/browser_switch --out_folder gen/chrome/browser/resources/browser_switch --namespace_rewrites cr_slider.SliderTick\|SliderTick cr.addSingletonGetter\|addSingletonGetter cr.addWebUIListener\|addWebUIListener cr.dispatchSimpleEvent\|dispatchSimpleEvent cr.icon.getFavicon\|getFavicon cr.icon.getImage\|getImage cr.isAndroid\|isAndroid cr.isChromeOS\|isChromeOS cr.isIOS\|isIOS cr.isLinux\|isLinux cr.isMac\|isMac cr.isWindows\|isWindows cr.png.convertImageSequenceToPng\|convertImageSequenceToPng cr.removeWebUIListener\|removeWebUIListener cr.sendWithPromise\|sendWithPromise cr.toastManager.getToastManager\|getToastManager cr.ui.FocusOutlineManager\|FocusOutlineManager cr.ui.FocusRowBehavior\|FocusRowBehavior cr.ui.focusWithoutInk\|focusWithoutInk cr.ui.Store\|Store Polymer.dom.flush\|flush Polymer.IronA11yAnnouncer\|IronA11yAnnouncer Polymer.PaperRippleBehavior\|PaperRippleBehavior Polymer.RenderStatus.afterNextRender\|afterNextRender Polymer.RenderStatus.beforeNextRender\|beforeNextRender --auto_imports third_party/polymer/v1_0/components-chromium/iron-a11y-announcer/iron-a11y-announcer.html\|IronA11yAnnouncer third_party/polymer/v1_0/components-chromium/paper-behaviors/paper-ripple-behavior.html\|PaperRippleBehavior ui/webui/resources/cr_elements/cr_scrollable_behavior.html\|CrScrollableBehavior ui/webui/resources/cr_elements/cr_radio_button/cr_radio_button_behavior.html\|CrRadioButtonBehavior ui/webui/resources/cr_elements/policy/cr_policy_pref_behavior.html\|CrPolicyPrefBehavior ui/webui/resources/html/cr/ui/focus_outline_manager.html\|FocusOutlineManager ui/webui/resources/html/cr/ui/focus_row_behavior.html\|FocusRowBehavior ui/webui/resources/html/cr/ui/focus_without_ink.html\|focusWithoutInk ui/webui/resources/html/find_shortcut_behavior.html\|FindShortcutBehavior ui/webui/resources/html/i18n_behavior.html\|I18nBehavior ui/webui/resources/html/list_property_update_behavior.html\|ListPropertyUpdateBehavior ui/webui/resources/html/load_time_data.html\|loadTimeData ui/webui/resources/html/parse_html_subset.html\|parseHtmlSubset ui/webui/resources/html/polymer.html\|Polymer,html ui/webui/resources/html/promise_resolver.html\|PromiseResolver ui/webui/resources/html/web_ui_listener_behavior.html\|WebUIListenerBehavior
[153/21294] python ../../../../../../../../../../src/qt5_webengine/2023-09-12/src/lib/qt5/qtwebengine/src/3rdparty/chromium/mojo/public/tools/bindings/mojom_bindings_generator.py -o gen verify --filelist=9642632244953993194..rsp -f gen/chrome/browser/resource_coordinator/mojo_bindings.deps_sources_list --depth ../../../../../../../../../../src/qt5_webengine/2023-09-12/src/lib/qt5/qtwebengine/src/3rdparty/chromium/
FAILED: gen/chrome/browser/resource_coordinator/lifecycle_unit_state.v 
python ../../../../../../../../../../src/qt5_webengine/2023-09-12/src/lib/qt5/qtwebengine/src/3rdparty/chromium/mojo/public/tools/bindings/mojom_bindings_generator.py -o gen verify --filelist=9642632244953993194..rsp -f gen/chrome/browser/resource_coordinator/mojo_bindings.deps_sources_list --depth ../../../../../../../../../../src/qt5_webengine/2023-09-12/src/lib/qt5/qtwebengine/src/3rdparty/chromium/
Traceback (most recent call last):
  File "/home/u/Desktop/sculpt_play/genode/depot/mewmew/bin/x86_64/qt5_webengine/2023-09-12.build/var/libcache/qt5_webengine/src/core/release/../../../../../../../../../../src/qt5_webengine/2023-09-12/src/lib/qt5/qtwebengine/src/3rdparty/chromium/mojo/public/tools/bindings/mojom_bindings_generator.py", line 43, in <module>
    from mojom.generate import template_expander
  File "/home/u/Desktop/sculpt_play/genode/depot/mewmew/src/qt5_webengine/2023-09-12/src/lib/qt5/qtwebengine/src/3rdparty/chromium/mojo/public/tools/mojom/mojom/generate/template_expander.py", line 13, in <module>
    import jinja2
  File "/home/u/Desktop/sculpt_play/genode/depot/mewmew/src/qt5_webengine/2023-09-12/src/lib/qt5/qtwebengine/src/3rdparty/chromium/third_party/jinja2/__init__.py", line 33, in <module>
    from jinja2.environment import Environment, Template
  File "/home/u/Desktop/sculpt_play/genode/depot/mewmew/src/qt5_webengine/2023-09-12/src/lib/qt5/qtwebengine/src/3rdparty/chromium/third_party/jinja2/environment.py", line 16, in <module>
    from jinja2.defaults import BLOCK_START_STRING, \
  File "/home/u/Desktop/sculpt_play/genode/depot/mewmew/src/qt5_webengine/2023-09-12/src/lib/qt5/qtwebengine/src/3rdparty/chromium/third_party/jinja2/defaults.py", line 32, in <module>
    from jinja2.tests import TESTS as DEFAULT_TESTS
  File "/home/u/Desktop/sculpt_play/genode/depot/mewmew/src/qt5_webengine/2023-09-12/src/lib/qt5/qtwebengine/src/3rdparty/chromium/third_party/jinja2/tests.py", line 13, in <module>
    from collections import Mapping
ImportError: cannot import name 'Mapping' from 'collections' (/usr/lib/python3.11/collections/__init__.py)
ninja: build stopped: subcommand failed.
make[10]: *** [Makefile.gn_run:346: run_ninja] Error 1
make[9]: *** [Makefile:82: sub-gn_run-pro-make_first] Error 2
make[8]: *** [Makefile:77: sub-core-make_first] Error 2
make[7]: *** [Makefile:46: sub-src] Error 2
make[6]: *** [/home/u/Desktop/sculpt_play/genode/depot/mewmew/src/qt5_webengine/2023-09-12/lib/mk/qt5_webengine.inc:31: built.tag] Error 2
make[5]: *** [var/libdeps:402: qt5_webengine.lib] Error 2
make[4]: *** [Makefile:336: gen_deps_and_build_targets] Error 2
make[3]: *** [/home/u/Desktop/sculpt_play/genode/tool/depot/mk/build_bin_archive:208: /home/u/Desktop/sculpt_play/genode/depot/mewmew/bin/x86_64/qt5_webengine/2023-09-12.build/bin] Error 1
make[2]: *** [/home/u/Desktop/sculpt_play/genode/depot/var/build.mk:12: mewmew/bin/x86_64/qt5_webengine/2023-09-12] Error 2
make[1]: *** [/home/u/Desktop/sculpt_play/genode/tool/depot/build:136: execute_generated_build_mk_file] Error 2
make: *** [tool/depot/create:59: build] Error 2

Edit: looks like the chromium/third_party/jinja2/tests.py python script depends on Python 2 (https://docs.python.org/2/library/collections.html#collections.Mapping), since collections.Mapping is missing from Python 3.

mewmew commented 1 year ago

Applying the following patch to https://github.com/cproc/qt5 (on top of the issue4863 branch) fixes the collections.Mapping issue at least.

commit 04242a3f1f49841cb1d430b8344a48666035ac99
Author: Robin Eklind
Date:   Wed Sep 13 01:52:56 2023 +0200

    jinja2: update to use dict (Python 3) instead of collections.Mapping (Python 2)

diff --git a/qtwebengine/src/3rdparty/chromium/third_party/jinja2/tests.py b/qtwebengine/src/3rdparty/chromium/third_party/jinja2/tests.py
index 0adc3d4dbc..b043d1fb0d 100644
--- a/qtwebengine/src/3rdparty/chromium/third_party/jinja2/tests.py
+++ b/qtwebengine/src/3rdparty/chromium/third_party/jinja2/tests.py
@@ -10,7 +10,6 @@
 """
 import operator
 import re
-from collections import Mapping
 from jinja2.runtime import Undefined
 from jinja2._compat import text_type, string_types, integer_types
 import decimal
@@ -84,7 +83,7 @@ def test_mapping(value):

     .. versionadded:: 2.6
     """
-    return isinstance(value, Mapping)
+    return isinstance(value, dict)

 def test_number(value):
mewmew commented 1 year ago

Upstream issue tracking update from Python 2 to Python 3 for generating ninja build files.

https://bugs.chromium.org/p/chromium/issues/detail?id=941669

mewmew commented 1 year ago

Updated the issue title to indicate that the issue is related to the requirement of using Python 2 to build qt5_webengine.

Python 2 was sunset in 2020.

From https://www.python.org/doc/sunset-python-2/

We are volunteers who make and take care of the Python programming language. We have decided that January 1, 2020, was the day that we sunset Python 2. That means that we will not improve it anymore after that day, even if someone finds a security problem in it. You should upgrade to Python 3 as soon as you can.

(Highlighted extract related to security issues not being patched or backported to Pyhon 2 anymore).

Hopefully the upstream issue (https://bugs.chromium.org/p/chromium/issues/detail?id=941669) will be able to mitigate this situation in the future.

Cheers, Robin