ungoogled-software / ungoogled-chromium-debian

Debian, Ubuntu, and others packaging for ungoogled-chromium
373 stars 49 forks source link

Could not find gn executable at: /ungoogled-chromium-debian/buildtools/linux64/gn OS: Ubuntu 22.04 #334

Open raphael10-collab opened 1 year ago

raphael10-collab commented 1 year ago

generate-ninja is installed :

raphy@raohy:~$ sudo apt-get install generate-ninja
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
generate-ninja is already the newest version (0.0~git20220118.0725d78-1).
0 upgraded, 0 newly installed, 0 to remove and 4 not upgraded.
raphy@raohy:~$ sudo apt-get install gn
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Note, selecting 'generate-ninja' instead of 'gn'
generate-ninja is already the newest version (0.0~git20220118.0725d78-1).
0 upgraded, 0 newly installed, 0 to remove and 4 not upgraded.
raphy@raohy:~$ 

But following these building steps: https://github.com/ungoogled-software/ungoogled-chromium-debian : UngoogleChromiumUbuntu22.04DesktopBuildingStepsFollowed.txt

I get this error:

gn.py: Could not find gn executable at: /home/raphy/ungoogled-chromium-debian/buildtools/linux64/gn

Other info:

O.S.: Ubuntu 22.04 Desktop
raphy@raohy:~/ungoogled-chromium-debian$ python3 --version
Python 3.10.12

How to make it work?

Ark74 commented 1 year ago

Interesting, I'm using sbuild to build on a chroot, it looks similar but not quite. This is the error I get.

...
WARNING: Skipping non-existant path: /<<PKGBUILDDIR>>/third_party/opus/src/doc/release.txt
WARNING: Skipping non-existant path: /<<PKGBUILDDIR>>/third_party/re2/src/CMakeLists.txt
WARNING: Skipping non-existant path: /<<PKGBUILDDIR>>/third_party/re2/src/re2/testing/re2_test.cc
WARNING: Skipping non-existant path: /<<PKGBUILDDIR>>/third_party/re2/src/util/pcre.h
WARNING: Skipping non-existant path: /<<PKGBUILDDIR>>/third_party/snappy/linux/snappy-stubs-public.h
WARNING: Skipping non-existant path: /<<PKGBUILDDIR>>/third_party/snappy/mac/snappy-stubs-public.h
WARNING: Skipping non-existant path: /<<PKGBUILDDIR>>/third_party/snappy/src/snappy-test.h
WARNING: Skipping non-existant path: /<<PKGBUILDDIR>>/third_party/snappy/win32/snappy-stubs-public.h
WARNING: Skipping non-existant path: /<<PKGBUILDDIR>>/third_party/zlib/deflate.c
INFO: Path has no substitutions: tools/md_browser/base.css
cp /<<PKGBUILDDIR>>/debian/shims/jsoncpp.gn build/linux/unbundle
build/linux/unbundle/replace_gn_files.py --system-libraries flac fontconfig freetype jsoncpp libdrm libevent libjpeg libpng libwebp libxml libxslt opus re2 snappy zlib 
make[1]: Leaving directory '/<<PKGBUILDDIR>>'
   debian/rules override_dh_auto_build
make[1]: Entering directory '/<<PKGBUILDDIR>>'
gn gen out/Release --args="build_with_tflite_lib=false chrome_pgo_phase=0 clang_use_chrome_plugins=false disable_fieldtrial_testing_config=true enable_hangout_services_extension=false enable_mdns=false enable_mse_mpeg2ts_stream_parser=true enable_nacl=false enable_reading_list=false enable_remoting=false enable_reporting=false enable_service_discovery=false enable_widevine=true exclude_unwind_tables=true google_api_key=\"\" google_default_client_id=\"\" google_default_client_secret=\"\" safe_browsing_mode=0 treat_warnings_as_errors=false use_official_google_api_keys=false use_unofficial_version_number=false  custom_toolchain=\"//build/toolchain/linux/unbundle:default\" host_toolchain=\"//build/toolchain/linux/unbundle:default\" use_custom_libcxx=false is_official_build=true is_debug=false symbol_level=1 blink_enable_generated_code_formatting=false is_cfi=false use_thin_lto=false use_sysroot=false use_gnome_keyring=false ffmpeg_branding=\"Chrome\" proprietary_codecs=true use_pulseaudio=true link_pulseaudio=true use_vaapi=true use_ozone=true use_goma=false enable_vr=false enable_iterator_debugging=false optimize_webui=true use_gio=true use_lld=true is_clang=true use_kerberos=false use_cups=true v8_enable_backtrace=true  perfetto_use_system_zlib=true use_system_lcms2=true use_system_libopenjpeg2=true use_system_libpng=true use_system_zlib=true use_system_libjpeg=true " --fail-on-unused-args
ERROR at //build/config/compiler/BUILD.gn:1418:22: Script returned non-zero exit code.
    clang_revision = exec_script("//tools/clang/scripts/update.py",
                     ^----------
Current dir: /<<PKGBUILDDIR>>/out/Release/
Command: python3 /<<PKGBUILDDIR>>/tools/clang/scripts/update.py --print-revision --verify-version=17
Returned 1 and printed out:

The expected clang version is llvmorg-17-init-8029-g27f27d15-3 but the actual version is 
Did you run "gclient sync"?

See //build/config/BUILDCONFIG.gn:333:3: which caused the file to be included.
  "//build/config/compiler:afdo",
  ^-----------------------------
make[1]: *** [debian/rules:142: override_dh_auto_build] Error 1
make[1]: Leaving directory '/<<PKGBUILDDIR>>'
make: *** [debian/rules:109: binary] Error 2
dpkg-buildpackage: error: fakeroot debian/rules binary subprocess returned exit status 2
--------------------------------------------------------------------------------
Build finished at 2023-10-04T00:10:45Z

Finished
--------

Seems to be an issue at https://commondatastorage.googleapis.com/chromium-browser-clang where the specified clang pre-compiled binaries is no longer available.

Checking...

Ark74 commented 1 year ago

Indeed,

CLANG_REVISION = 'llvmorg-17-init-8029-g27f27d15'
CLANG_SUB_REVISION = 3

to

CLANG_REVISION = 'llvmorg-17-init-857-g4f2fde78'
CLANG_SUB_REVISION = 1

at tools/clang/scripts/update.py

That should get you pass that line. Regards.

Ark74 commented 1 year ago

Oh well, later on seems that we need a specific version otherwise it will fail.

[14976/54516] CXX obj/chrome/common/channel_info/channel_info.o
FAILED: obj/chrome/common/channel_info/channel_info.o 
clang++-14 -MMD -MF obj/chrome/common/channel_info/channel_info.o.d -DUSE_UDEV -DUSE_AURA=1 -DUSE_GLIB=1 -DUSE_OZONE=1 -DOFFICIAL_BUILD -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D_FORTIFY_SOURCE=2 -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -DNO_UNWIND_TABLES -D_GNU_SOURCE -DCR_CLANG_REVISION=\"llvmorg-17-init-857-g4f2fde78-1\" -DNDEBUG -DNVALGRIND -DDYNAMIC_ANNOTATIONS_ENABLED=0 -DGLIB_VERSION_MAX_ALLOWED=GLIB_VERSION_2_56 -DGLIB_VERSION_MIN_REQUIRED=GLIB_VERSION_2_56 -DBASE_USE_PERFETTO_CLIENT_LIBRARY=1 -I../.. -Igen -I../../third_party/perfetto/include -Igen/third_party/perfetto/build_config -Igen/third_party/perfetto -Igen/shim_headers/zlib_shim -Igen/shim_headers/jsoncpp_shim -Igen/shim_headers/libevent_shim -I../../third_party/abseil-cpp -I../../third_party/boringssl/src/include -I../../third_party/protobuf/src -Igen/protoc_out -Wall -Wextra -Wimplicit-fallthrough -Wextra-semi -Wunreachable-code-aggressive -Wthread-safety -Wno-missing-field-initializers -Wno-unused-parameter -Wno-psabi -Wloop-analysis -Wno-unneeded-internal-declaration -Wenum-compare-conditional -Wno-ignored-pragma-optimize -Wno-deprecated-builtins -Wno-bitfield-constant-conversion -Wno-deprecated-this-capture -Wshadow -fno-delete-null-pointer-checks -fno-ident -fno-strict-aliasing --param=ssp-buffer-size=4 -fstack-protector -fno-unwind-tables -fno-asynchronous-unwind-tables -fPIC -pthread -fcolor-diagnostics -fmerge-all-constants -fcrash-diagnostics-dir=../../tools/clang/crashreports -mllvm -instcombine-lower-dbg-declare=0 -ffp-contract=off -m64 -msse3 -ffile-compilation-dir=. -no-canonical-prefixes -ftrivial-auto-var-init=pattern -O2 -fdata-sections -ffunction-sections -fno-unique-section-names -fno-omit-frame-pointer -gdwarf-4 -g1 -gdwarf-aranges -fvisibility=hidden -Wheader-hygiene -Wstring-conversion -Wtautological-overlap-compare -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include -DPROTOBUF_ALLOW_DEPRECATED=1 -std=c++20 -Wno-trigraphs -gsimple-template-names -fno-exceptions -fno-rtti -fvisibility-inlines-hidden -Wdate-time -D_FORTIFY_SOURCE=2 -O2 -ffile-prefix-map=/<<PKGBUILDDIR>>=. -fstack-protector-strong -Wformat -Werror=format-security -Wno-unknown-warning-option -Wno-deprecated-declarations -c ../../chrome/common/channel_info.cc -o obj/chrome/common/channel_info/channel_info.o
In file included from ../../chrome/common/channel_info.cc:7:
../../components/version_info/version_info.h:30:23: error: constexpr function's return type 'std::string' (aka 'basic_string<char>') is not a literal type
constexpr std::string GetProductName() {

So yeah, we need to track the compatible version and try again, and so on as many times an issue is found.

raphael10-collab commented 1 year ago

Hi @Ark74 ! What should I do then in order to make it work?

Ark74 commented 1 year ago

So far, no idea. I also would like to build for a jammy base, but 'til a developer give us some light on the matter, I'm just like everyone else that is trying and failing. :laughing:

Ark74 commented 12 months ago

@raphael10-collab based on the open PR #328 there is already a PPA with the latest version.

I've been testing it and it works and builds correctly, thanks to @jhonny-oliveira

Regards.

raphael10-collab commented 12 months ago

@Ark74 Thank you .

Actually I need the source code, and the correct working way to build it

jhonny-oliveira commented 12 months ago

@Ark74 Thank you .

Actually I need the source code, and the correct working way to build it

Check this out: https://github.com/ungoogled-software/ungoogled-chromium-debian/pull/328#issuecomment-1690539514

You might no longer need the PRs (they were already merged) or you might needs new ones (with new fixes).

For a matter of alignment, I usually download the source from here: https://packages.debian.org/sid/chromium.

Hope this helps! Cheers!

raphael10-collab commented 11 months ago

Hi @jhonny-oliveira !

Thanks for the links.

I downloaded .deb file from here :https://packages.debian.org/sid/amd64/chromium/download

But during the source extraction I get this error:

raphy@raohy:~/ungoogled-chromium-debian$ dpkg-source -x /home/raphy/Downloads/chromium_118.0.5993.70-1_amd64.deb
dpkg-source: error: syntax error in /home/raphy/Downloads/chromium_118.0.5993.70-1_amd64.deb at line 2: line with unknown format (not field-colon-value)
Ark74 commented 11 months ago

@jhonny-oliveira I was able to build using your PPA source in amd64, for ppc64el I added this patch and some small changes in our helper, it is still building.

armhf and arm64 were failing, I'll test back again once ppc64el finish, if the whole set succeeds it's possible we could integrate it on the trisquel repo.

Cheers!

Ark74 commented 11 months ago

@raphael10-collab dpkg-source -x works with tarballs and .dsc files, .deb are binaries, they don't have source in it, unless it's stated the binary contains the source, but that's not the case for this package.

To get the source code go at the bottom of the page in:

Download the 3 files, 2 tar.xz and 1 dsc on the same directory, then do,

dpkg-source -x chromium_118.0.5993.70-1.dsc

Then you'll have the sid chromium source (not ungoogled-chromium), keep in mind that is very likely Debian Sid dependencies won't be compatible with Jammy's ones, as Sid is the "unstable" branch.

Regards.

raphael10-collab commented 11 months ago

@Ark74 Thank you Luis

Till the sid chromium source seems fine:

raphy@raohy:~/ungoogled-chromium-debian/convert$ wget deb.debian.org/debian/pool/main/c/chromium/chromium_118.0.5993.70-1.dsc
--2023-10-12 21:10:35--  http://deb.debian.org/debian/pool/main/c/chromium/chromium_118.0.5993.70-1.dsc
Resolving deb.debian.org (deb.debian.org)... 2a04:4e42:39::644, 151.101.242.132
Connecting to deb.debian.org (deb.debian.org)|2a04:4e42:39::644|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 3659 (3,6K) [text/prs.lines.tag]
Saving to: ‘chromium_118.0.5993.70-1.dsc’

chromium_118.0.5993.70-1.dsc       100%[==============================================================>]   3,57K  --.-KB/s    in 0s      

2023-10-12 21:10:35 (386 MB/s) - ‘chromium_118.0.5993.70-1.dsc’ saved [3659/3659]

raphy@raohy:~/ungoogled-chromium-debian/convert$ wget deb.debian.org/debian/pool/main/c/chromium/chromium_118.0.5993.70.orig.tar.xz
--2023-10-12 21:10:47--  http://deb.debian.org/debian/pool/main/c/chromium/chromium_118.0.5993.70.orig.tar.xz
Resolving deb.debian.org (deb.debian.org)... 2a04:4e42:7d::644, 151.101.242.132
Connecting to deb.debian.org (deb.debian.org)|2a04:4e42:7d::644|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 771394968 (736M) [application/x-xz]
Saving to: ‘chromium_118.0.5993.70.orig.tar.xz’

chromium_118.0.5993.70.orig.tar.xz 100%[==============================================================>] 735,66M   110MB/s    in 7,0s    

2023-10-12 21:10:54 (106 MB/s) - ‘chromium_118.0.5993.70.orig.tar.xz’ saved [771394968/771394968]

raphy@raohy:~/ungoogled-chromium-debian/convert$ wget deb.debian.org/debian/pool/main/c/chromium/chromium_118.0.5993.70-1.debian.tar.xz
--2023-10-12 21:11:01--  http://deb.debian.org/debian/pool/main/c/chromium/chromium_118.0.5993.70-1.debian.tar.xz
Resolving deb.debian.org (deb.debian.org)... 2a04:4e42:39::644, 151.101.242.132
Connecting to deb.debian.org (deb.debian.org)|2a04:4e42:39::644|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 389388 (380K) [application/x-xz]
Saving to: ‘chromium_118.0.5993.70-1.debian.tar.xz’

chromium_118.0.5993.70-1.debian.ta 100%[==============================================================>] 380,26K  --.-KB/s    in 0,02s   

2023-10-12 21:11:01 (19,9 MB/s) - ‘chromium_118.0.5993.70-1.debian.tar.xz’ saved [389388/389388]

raphy@raohy:~/ungoogled-chromium-debian/convert$ ls -lah
total 737M
drwxrwxr-x 2 raphy raphy 4,0K ott 12 21:11 .
drwxrwxr-x 6 raphy raphy 4,0K ott 12 17:08 ..
-rw-rw-r-- 1 raphy raphy 381K ott 11 05:59 chromium_118.0.5993.70-1.debian.tar.xz
-rw-rw-r-- 1 raphy raphy 3,6K ott 11 05:59 chromium_118.0.5993.70-1.dsc
-rw-rw-r-- 1 raphy raphy 736M ott 11 05:59 chromium_118.0.5993.70.orig.tar.xz
-rwxrwxr-x 1 raphy raphy 2,3K ott 12 17:08 compare.sh
-rwxrwxr-x 1 raphy raphy 1,6K ott 12 17:08 editcontrol.pl
-rw-rw-r-- 1 raphy raphy  18K ott 12 21:07 Makefile
-rw-rw-r-- 1 raphy raphy 3,7K ott 12 17:08 README.md
-rw-rw-r-- 1 raphy raphy 1,5K ott 12 17:08 replace-name.pl
raphy@raohy:~/ungoogled-chromium-debian/convert$ dpkg-source -x chromium_118.0.5993.70-1.dsc
gpgv: Signature made mer 11 ott 2023, 05:48:02 CEST
gpgv:                using RSA key 6E58CA48BEAEF11C0345205EEC3C80BC57319B7E
gpgv: Can't check signature: No public key
dpkg-source: warning: cannot verify signature ./chromium_118.0.5993.70-1.dsc
dpkg-source: info: extracting chromium in chromium-118.0.5993.70
dpkg-source: info: unpacking chromium_118.0.5993.70.orig.tar.xz
dpkg-source: info: unpacking chromium_118.0.5993.70-1.debian.tar.xz
dpkg-source: info: using patch list from debian/patches/series
dpkg-source: info: applying debianization/manpage.patch
dpkg-source: info: applying debianization/sandbox.patch
dpkg-source: info: applying debianization/master-preferences.patch
dpkg-source: info: applying debianization/clang-version.patch
dpkg-source: info: applying fixes/ps-print.patch
dpkg-source: info: applying fixes/widevine-revision.patch
dpkg-source: info: applying fixes/widevine-locations.patch
dpkg-source: info: applying fixes/brandversion-construct.patch
dpkg-source: info: applying fixes/SkColor4f-init.patch
dpkg-source: info: applying fixes/gcc13-with-clang14.patch
dpkg-source: info: applying fixes/gcc13-headers.patch
dpkg-source: info: applying fixes/rust-clanglib.patch
dpkg-source: info: applying fixes/material-utils.patch
dpkg-source: info: applying fixes/perfetto.patch
dpkg-source: info: applying fixes/v8-compressed-ptrs.patch
dpkg-source: info: applying upstream/mojo.patch
dpkg-source: info: applying upstream/sensor-reading.patch
dpkg-source: info: applying upstream/lweight.patch
dpkg-source: info: applying upstream/freetype.patch
dpkg-source: info: applying upstream/sizet.patch
dpkg-source: info: applying disable/tests.patch
dpkg-source: info: applying disable/unrar.patch
dpkg-source: info: applying disable/signin.patch
dpkg-source: info: applying disable/android.patch
dpkg-source: info: applying disable/catapult.patch
dpkg-source: info: applying disable/font-tests.patch
dpkg-source: info: applying disable/swiftshader.patch
dpkg-source: info: applying disable/swiftshader-2.patch
dpkg-source: info: applying disable/google-api-warning.patch
dpkg-source: info: applying disable/third-party-cookies.patch
dpkg-source: info: applying disable/driver-chrome-path.patch
dpkg-source: info: applying disable/widevine-cdm-cu.patch
dpkg-source: info: applying disable/angle-perftests.patch
dpkg-source: info: applying disable/clang-version-check.patch
dpkg-source: info: applying disable/privacy-sandbox.patch
dpkg-source: info: applying system/icu-shim.patch
dpkg-source: info: applying system/jpeg.patch
dpkg-source: info: applying system/zlib.patch
dpkg-source: info: applying system/event.patch
dpkg-source: info: applying system/openjpeg.patch
dpkg-source: info: applying system/convertutf.patch
dpkg-source: info: applying system/clang-format.patch
dpkg-source: info: applying system/opus.patch
dpkg-source: info: applying system/eu-strip.patch
dpkg-source: info: applying system/rapidjson.patch
dpkg-source: info: applying system/rollup.patch
dpkg-source: info: applying bookworm/clang-attribs.patch
dpkg-source: info: applying bookworm/typename.patch
dpkg-source: warning: diff 'chromium-118.0.5993.70/debian/patches/bookworm/typename.patch' patches file chromium-118.0.5993.70/components/optimization_guide/core/tflite_model_executor.h more than once
dpkg-source: warning: diff 'chromium-118.0.5993.70/debian/patches/bookworm/typename.patch' patches file chromium-118.0.5993.70/ui/base/interaction/interactive_test_internal.h more than once
dpkg-source: info: applying bookworm/struct-ctor.patch
dpkg-source: info: applying bookworm/structured-binding-scope-bug.patch
dpkg-source: info: applying bookworm/stringpiece3.patch
dpkg-source: info: applying bookworm/initialize-const-ctor.patch
dpkg-source: info: applying i386/support-i386.patch
dpkg-source: info: applying i386/angle-lockfree.patch
dpkg-source: info: applying ppc64le/sandbox/0001-linux-seccomp-bpf-ppc64-glibc-workaround-in-SIGSYS-h.patch
dpkg-source: info: applying ppc64le/sandbox/0001-sandbox-Enable-seccomp_bpf-for-ppc64.patch
dpkg-source: info: applying ppc64le/sandbox/0001-services-service_manager-sandbox-linux-Fix-TCGETS-de.patch
dpkg-source: info: applying ppc64le/sandbox/0001-sandbox-linux-bpf_dsl-Update-syscall-ranges-for-ppc6.patch
dpkg-source: info: applying ppc64le/sandbox/0001-sandbox-linux-Implement-partial-support-for-ppc64-sy.patch
dpkg-source: info: applying ppc64le/sandbox/0001-sandbox-linux-Update-IsSyscallAllowed-in-broker_proc.patch
dpkg-source: info: applying ppc64le/sandbox/0001-sandbox-linux-Update-syscall-helpers-lists-for-ppc64.patch
dpkg-source: info: applying ppc64le/sandbox/0002-sandbox-linux-bpf_dsl-Modify-seccomp_macros-to-add-s.patch
dpkg-source: info: applying ppc64le/sandbox/0003-sandbox-linux-system_headers-Update-linux-seccomp-he.patch
dpkg-source: info: applying ppc64le/sandbox/0004-sandbox-linux-system_headers-Update-linux-signal-hea.patch
dpkg-source: info: applying ppc64le/sandbox/0005-sandbox-linux-seccomp-bpf-Add-ppc64-syscall-stub.patch
dpkg-source: info: applying ppc64le/sandbox/0005-sandbox-linux-update-unit-test-for-ppc64.patch
dpkg-source: info: applying ppc64le/sandbox/0006-sandbox-linux-disable-timedwait-time64-ppc64.patch
dpkg-source: info: applying ppc64le/sandbox/0007-sandbox-linux-add-ppc64-stat.patch
dpkg-source: info: applying ppc64le/sandbox/Sandbox-linux-services-credentials.cc-PPC.patch
dpkg-source: info: applying ppc64le/sandbox/0008-sandbox-fix-ppc64le-glibc234.patch
dpkg-source: info: applying ppc64le/third_party/0001-third_party-angle-Include-missing-header-cstddef-in-.patch
dpkg-source: info: applying ppc64le/third_party/0001-Add-PPC64-support-for-boringssl.patch
dpkg-source: info: applying ppc64le/third_party/0001-third_party-libvpx-Properly-generate-gni-on-ppc64.patch
dpkg-source: info: applying ppc64le/third_party/0001-third_party-lss-Don-t-look-for-mmap2-on-ppc64.patch
dpkg-source: info: applying ppc64le/third_party/0001-third_party-pffft-Include-altivec.h-on-ppc64-with-SI.patch
dpkg-source: info: applying ppc64le/third_party/0002-third_party-lss-kernel-structs.patch
dpkg-source: info: applying ppc64le/webrtc/Rtc_base-system-arch.h-PPC.patch
dpkg-source: info: applying ppc64le/crashpad/0002-Include-cstddef-to-fix-build.patch
dpkg-source: info: applying ppc64le/third_party/0004-third_party-crashpad-port-curl-transport-ppc64.patch
dpkg-source: info: applying ppc64le/workarounds/HACK-third_party-libvpx-use-generic-gnu.patch
dpkg-source: info: applying ppc64le/workarounds/HACK-debian-clang-disable-skia-musttail.patch
dpkg-source: info: applying ppc64le/libaom/0001-Add-ppc64-target-to-libaom.patch
dpkg-source: info: applying ppc64le/libaom/0001-Add-pregenerated-config-for-libaom-on-ppc64.patch
dpkg-source: info: applying ppc64le/third_party/0002-third_party-libvpx-Remove-bad-ppc64-config.patch
dpkg-source: info: applying ppc64le/third_party/0002-third-party-boringssl-add-generated-files.patch
dpkg-source: info: applying ppc64le/third_party/0003-third_party-libvpx-Add-ppc64-generated-config.patch
dpkg-source: info: applying ppc64le/third_party/0003-third_party-ffmpeg-Add-ppc64-generated-config.patch
dpkg-source: info: applying ppc64le/third_party/0004-third_party-libvpx-work-around-ambiguous-vsx.patch
dpkg-source: info: applying ppc64le/third_party/skia-vsx-instructions.patch
dpkg-source: info: applying ppc64le/ffmpeg/0001-Add-support-for-ppc64.patch
dpkg-source: info: applying ppc64le/breakpad/0001-Implement-support-for-ppc64-on-Linux.patch
dpkg-source: info: applying ppc64le/crashpad/0001-Implement-support-for-PPC64-on-Linux.patch
dpkg-source: info: applying ppc64le/database/0001-Properly-detect-little-endian-PPC64-systems.patch
dpkg-source: info: applying ppc64le/third_party/0001-Force-baseline-POWER8-AltiVec-VSX-CPU-features-when-.patch
dpkg-source: info: applying ppc64le/fixes/fix-breakpad-compile.patch
dpkg-source: info: applying ppc64le/fixes/fix-partition-alloc-compile.patch
dpkg-source: info: applying ppc64le/v8/0002-Add-ppc64-trap-instructions.patch
dpkg-source: info: applying ppc64le/third_party/0001-Fix-highway-ppc-hwcap.patch
dpkg-source: info: applying ppc64le/third_party/0001-Add-PPC64-support-for-libdav1d.patch
dpkg-source: info: applying ppc64le/third_party/0001-Fix-libdav1d-compilation-on-clang-ppc.patch
dpkg-source: info: applying ppc64le/sandbox/fix-ppc64-linux-syscalls-headers.patch
dpkg-source: info: applying ppc64le/third_party/0003-thirdparty-fix-dav1d-gn.patch
dpkg-source: info: applying ppc64le/third_party/use-sysconf-page-size-on-ppc64.patch
dpkg-source: info: applying ppc64le/third_party/dawn-fix-typos.patch
dpkg-source: info: applying ppc64le/third_party/dawn-fix-ppc64le-detection.patch
dpkg-source: info: applying ppc64le/fixes/fix-unknown-warning-option-messages.diff
dpkg-source: info: applying ppc64le/third_party/0002-Highway-disable-128-bit-vsx.patch
dpkg-source: info: applying ungoogled/core/ungoogled-chromium/disable-web-environment-integrity.patch

raphy@raohy:~/ungoogled-chromium-debian/convert$ ls -lah
total 737M
drwxrwxr-x  3 raphy raphy 4,0K ott 12 21:12 .
drwxrwxr-x  6 raphy raphy 4,0K ott 12 17:08 ..
drwxrwxr-x 55 raphy raphy 4,0K ott 12 21:12 chromium-118.0.5993.70
-rw-rw-r--  1 raphy raphy 381K ott 11 05:59 chromium_118.0.5993.70-1.debian.tar.xz
-rw-rw-r--  1 raphy raphy 3,6K ott 11 05:59 chromium_118.0.5993.70-1.dsc
-rw-rw-r--  1 raphy raphy 736M ott 11 05:59 chromium_118.0.5993.70.orig.tar.xz
-rwxrwxr-x  1 raphy raphy 2,3K ott 12 17:08 compare.sh
-rwxrwxr-x  1 raphy raphy 1,6K ott 12 17:08 editcontrol.pl
-rw-rw-r--  1 raphy raphy  18K ott 12 21:07 Makefile
-rw-rw-r--  1 raphy raphy 3,7K ott 12 17:08 README.md
-rw-rw-r--  1 raphy raphy 1,5K ott 12 17:08 replace-name.pl

But the make phase asks for rule to series.add : https://github.com/ungoogled-software/ungoogled-chromium-debian/pull/328#issuecomment-1690539514 :

raphy@raohy:~/ungoogled-chromium-debian/convert$ make
make: *** No rule to make target '/home/raphy/src/ungoogled-chromium/patches/series', needed by 'series.add'.  Stop.
iskunk commented 11 months ago

Hi @Ark74, could you elaborate on why ppc64el needs ThinLTO disabled? I can integrate such a change into the XtraDeb source if needed.

(Please note that @jhonny-oliveira and I are still working on the automation to get ungoogled-chromium converted and updated automatically. Updates, let alone security updates, will be delayed until we have that in place. We will also be releasing the script we use to perform the Ubuntu conversion from the Debian source.)

@raphael10-collab, note that the files under ungoogled-chromium-debian/convert/ have their own documentation and procedure for use. The conversion framework cannot be used out of the box; there is some setup you will need to perform first.

Ark74 commented 11 months ago

@iskunk I did several testing, and I always hit the same error, something along the lines:

....
test -f   ungoogled-domain-substitution.orig.tar \
|| debian/ungoogled-domain-substitution.sh
Creating backup archive ...
Applying ungoogled-chromium domain substitution to 11885 files ...
gn gen out/Release --args="clang_use_chrome_plugins=false rust_sysroot_absolute=\"/usr\" rustc_version=\"rustc 1.66.1 (90743e729 2023-01-10) (built from a source tarball)\" enable_rust=false clang_base_path=\"/usr/lib/llvm-\" clang_version=\"""\" host_toolchain=\"//build/toolchain/linux/unbundle:default\" custom_toolchain=\"//build/toolchain/linux/unbundle:default\" is_debug=false use_goma=false use_sysroot=false use_libjpeg_turbo=true use_custom_libcxx=false use_unofficial_version_number=false enable_vr=false enable_nacl=false enable_swiftshader=false dawn_use_swiftshader=false build_dawn_tests=false enable_reading_list=false enable_iterator_debugging=false enable_hangout_services_extension=false angle_has_histograms=false build_angle_perftests=false treat_warnings_as_errors=false use_qt=false is_cfi=false use_thin_lto=true chrome_pgo_phase=0  use_gio=true is_official_build=true symbol_level=0 use_pulseaudio=true link_pulseaudio=true rtc_use_pipewire=true icu_use_data_file=true enable_widevine=true v8_enable_backtrace=true use_system_zlib=true use_system_lcms2=true use_system_libjpeg=true use_system_libpng=true use_system_freetype=true use_system_libopenjpeg2=true  proprietary_codecs=true ffmpeg_branding=\"Chrome\" disable_fieldtrial_testing_config=true  build_with_tflite_lib=false chrome_pgo_phase=0 clang_use_chrome_plugins=false disable_fieldtrial_testing_config=true enable_hangout_services_extension=false enable_mdns=false enable_nacl=false enable_reading_list=false enable_remoting=false enable_reporting=false enable_service_discovery=false enable_widevine=true exclude_unwind_tables=true google_api_key=\"\" google_default_client_id=\"\" google_default_client_secret=\"\" safe_browsing_mode=0 treat_warnings_as_errors=false use_official_google_api_keys=false use_unofficial_version_number=false "
ERROR at //build/config/compiler/BUILD.gn:696:5: Assertion failed.
    assert(use_lld, "LTO is only supported with lld")
    ^-----
LTO is only supported with lld
See //build/config/BUILDCONFIG.gn:333:3: which caused the file to be included.
  "//build/config/compiler:afdo",
  ^-----------------------------
make[1]: *** [debian/rules:164: override_dh_auto_build-arch] Error 1
make[1]: Leaving directory '/<<PKGBUILDDIR>>'
make: *** [debian/rules:127: binary] Error 2
dpkg-buildpackage: error: debian/rules binary subprocess returned exit status 2
--------------------------------------------------------------------------------
....

After not finding much information that Debian developers have decided to disable lto on chromium Debian#1015367 and comparing build environments for chromium and ungoogled-chromium I found that thin-lto is disabled in chromium, just by disabling thin-lto ppc64el started to build fine without any other change.

arm64(/hf) had been failing, since I've updated to not use thin-lto they seem will finish building, so I'm looking to disable to all but amd64.

ATTOW,

Which are the most progress or any of those archs.

You may be able to confirm my findings on the PPA build nodes. About the PPA, and current stage, it's ok, I'm still testing this out, I'll wait it gets kind of stable before any move.

Thank you guys for all the work you are putting here. Regards.

raphael10-collab commented 11 months ago

I've restarted from scratch:

raphy@raohy:~/ungoogled-chromium$ wget deb.debian.org/debian/pool/main/c/chromium/chromium_118.0.5993.70.orig.tar.xz
--2023-10-18 09:31:25--  http://deb.debian.org/debian/pool/main/c/chromium/chromium_118.0.5993.70.orig.tar.xz
Resolving deb.debian.org (deb.debian.org)... 2a04:4e42:39::644, 151.101.242.132
Connecting to deb.debian.org (deb.debian.org)|2a04:4e42:39::644|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 771394968 (736M) [application/x-xz]
Saving to: ‘chromium_118.0.5993.70.orig.tar.xz’

chromium_118.0.5993.70.orig.tar.xz                                100%[===========================================================================================================================================================>] 735,66M   110MB/s    in 6,7s    

2023-10-18 09:31:31 (110 MB/s) - ‘chromium_118.0.5993.70.orig.tar.xz’ saved [771394968/771394968]

raphy@raohy:~/ungoogled-chromium$ wget deb.debian.org/debian/pool/main/c/chromium/chromium_118.0.5993.70-1.debian.tar.xz
--2023-10-18 09:31:36--  http://deb.debian.org/debian/pool/main/c/chromium/chromium_118.0.5993.70-1.debian.tar.xz
Resolving deb.debian.org (deb.debian.org)... 2a04:4e42:7d::644, 151.101.242.132
Connecting to deb.debian.org (deb.debian.org)|2a04:4e42:7d::644|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 389388 (380K) [application/x-xz]
Saving to: ‘chromium_118.0.5993.70-1.debian.tar.xz’

chromium_118.0.5993.70-1.debian.tar.xz                            100%[===========================================================================================================================================================>] 380,26K  --.-KB/s    in 0,07s   

2023-10-18 09:31:36 (5,32 MB/s) - ‘chromium_118.0.5993.70-1.debian.tar.xz’ saved [389388/389388]

raphy@raohy:~/ungoogled-chromium$ ls -lah
total 737M
drwxrwxr-x  2 raphy raphy 4,0K ott 18 09:31 .
drwxr-x--- 65 raphy raphy 4,0K ott 18 09:30 ..
-rw-rw-r--  1 raphy raphy 381K ott 11 05:59 chromium_118.0.5993.70-1.debian.tar.xz
-rw-rw-r--  1 raphy raphy 3,6K ott 11 05:59 chromium_118.0.5993.70-1.dsc
-rw-rw-r--  1 raphy raphy 736M ott 11 05:59 chromium_118.0.5993.70.orig.tar.xz

raphy@raohy:~/ungoogled-chromium$ dpkg-source -x chromium_118.0.5993.70-1.dsc 
gpgv: Signature made mer 11 ott 2023, 05:48:02 CEST
gpgv:                using RSA key 6E58CA48BEAEF11C0345205EEC3C80BC57319B7E
gpgv: Can't check signature: No public key
dpkg-source: warning: cannot verify signature ./chromium_118.0.5993.70-1.dsc
dpkg-source: info: extracting chromium in chromium-118.0.5993.70
dpkg-source: info: unpacking chromium_118.0.5993.70.orig.tar.xz
dpkg-source: info: unpacking chromium_118.0.5993.70-1.debian.tar.xz
dpkg-source: info: using patch list from debian/patches/series
dpkg-source: info: applying debianization/manpage.patch
dpkg-source: info: applying debianization/sandbox.patch
dpkg-source: info: applying debianization/master-preferences.patch
dpkg-source: info: applying debianization/clang-version.patch
dpkg-source: info: applying fixes/ps-print.patch
dpkg-source: info: applying fixes/widevine-revision.patch
dpkg-source: info: applying fixes/widevine-locations.patch
dpkg-source: info: applying fixes/brandversion-construct.patch
dpkg-source: info: applying fixes/SkColor4f-init.patch
dpkg-source: info: applying fixes/gcc13-with-clang14.patch
dpkg-source: info: applying fixes/gcc13-headers.patch
dpkg-source: info: applying fixes/rust-clanglib.patch
dpkg-source: info: applying fixes/material-utils.patch
dpkg-source: info: applying fixes/perfetto.patch
dpkg-source: info: applying fixes/v8-compressed-ptrs.patch
dpkg-source: info: applying upstream/mojo.patch
dpkg-source: info: applying upstream/sensor-reading.patch
dpkg-source: info: applying upstream/lweight.patch
dpkg-source: info: applying upstream/freetype.patch
dpkg-source: info: applying upstream/sizet.patch
dpkg-source: info: applying disable/tests.patch
dpkg-source: info: applying disable/unrar.patch
dpkg-source: info: applying disable/signin.patch
dpkg-source: info: applying disable/android.patch
dpkg-source: info: applying disable/catapult.patch
dpkg-source: info: applying disable/font-tests.patch
dpkg-source: info: applying disable/swiftshader.patch
dpkg-source: info: applying disable/swiftshader-2.patch
dpkg-source: info: applying disable/google-api-warning.patch
dpkg-source: info: applying disable/third-party-cookies.patch
dpkg-source: info: applying disable/driver-chrome-path.patch
dpkg-source: info: applying disable/widevine-cdm-cu.patch
dpkg-source: info: applying disable/angle-perftests.patch
dpkg-source: info: applying disable/clang-version-check.patch
dpkg-source: info: applying disable/privacy-sandbox.patch
dpkg-source: info: applying system/icu-shim.patch
dpkg-source: info: applying system/jpeg.patch
dpkg-source: info: applying system/zlib.patch
dpkg-source: info: applying system/event.patch
dpkg-source: info: applying system/openjpeg.patch
dpkg-source: info: applying system/convertutf.patch
dpkg-source: info: applying system/clang-format.patch
dpkg-source: info: applying system/opus.patch
dpkg-source: info: applying system/eu-strip.patch
dpkg-source: info: applying system/rapidjson.patch
dpkg-source: info: applying system/rollup.patch
dpkg-source: info: applying bookworm/clang-attribs.patch
dpkg-source: info: applying bookworm/typename.patch
dpkg-source: warning: diff 'chromium-118.0.5993.70/debian/patches/bookworm/typename.patch' patches file chromium-118.0.5993.70/components/optimization_guide/core/tflite_model_executor.h more than once
dpkg-source: warning: diff 'chromium-118.0.5993.70/debian/patches/bookworm/typename.patch' patches file chromium-118.0.5993.70/ui/base/interaction/interactive_test_internal.h more than once
dpkg-source: info: applying bookworm/struct-ctor.patch
dpkg-source: info: applying bookworm/structured-binding-scope-bug.patch
dpkg-source: info: applying bookworm/stringpiece3.patch
dpkg-source: info: applying bookworm/initialize-const-ctor.patch
dpkg-source: info: applying i386/support-i386.patch
dpkg-source: info: applying i386/angle-lockfree.patch
dpkg-source: info: applying ppc64le/sandbox/0001-linux-seccomp-bpf-ppc64-glibc-workaround-in-SIGSYS-h.patch
dpkg-source: info: applying ppc64le/sandbox/0001-sandbox-Enable-seccomp_bpf-for-ppc64.patch
dpkg-source: info: applying ppc64le/sandbox/0001-services-service_manager-sandbox-linux-Fix-TCGETS-de.patch
dpkg-source: info: applying ppc64le/sandbox/0001-sandbox-linux-bpf_dsl-Update-syscall-ranges-for-ppc6.patch
dpkg-source: info: applying ppc64le/sandbox/0001-sandbox-linux-Implement-partial-support-for-ppc64-sy.patch
dpkg-source: info: applying ppc64le/sandbox/0001-sandbox-linux-Update-IsSyscallAllowed-in-broker_proc.patch
dpkg-source: info: applying ppc64le/sandbox/0001-sandbox-linux-Update-syscall-helpers-lists-for-ppc64.patch
dpkg-source: info: applying ppc64le/sandbox/0002-sandbox-linux-bpf_dsl-Modify-seccomp_macros-to-add-s.patch
dpkg-source: info: applying ppc64le/sandbox/0003-sandbox-linux-system_headers-Update-linux-seccomp-he.patch
dpkg-source: info: applying ppc64le/sandbox/0004-sandbox-linux-system_headers-Update-linux-signal-hea.patch
dpkg-source: info: applying ppc64le/sandbox/0005-sandbox-linux-seccomp-bpf-Add-ppc64-syscall-stub.patch
dpkg-source: info: applying ppc64le/sandbox/0005-sandbox-linux-update-unit-test-for-ppc64.patch
dpkg-source: info: applying ppc64le/sandbox/0006-sandbox-linux-disable-timedwait-time64-ppc64.patch
dpkg-source: info: applying ppc64le/sandbox/0007-sandbox-linux-add-ppc64-stat.patch
dpkg-source: info: applying ppc64le/sandbox/Sandbox-linux-services-credentials.cc-PPC.patch
dpkg-source: info: applying ppc64le/sandbox/0008-sandbox-fix-ppc64le-glibc234.patch
dpkg-source: info: applying ppc64le/third_party/0001-third_party-angle-Include-missing-header-cstddef-in-.patch
dpkg-source: info: applying ppc64le/third_party/0001-Add-PPC64-support-for-boringssl.patch
dpkg-source: info: applying ppc64le/third_party/0001-third_party-libvpx-Properly-generate-gni-on-ppc64.patch
dpkg-source: info: applying ppc64le/third_party/0001-third_party-lss-Don-t-look-for-mmap2-on-ppc64.patch
dpkg-source: info: applying ppc64le/third_party/0001-third_party-pffft-Include-altivec.h-on-ppc64-with-SI.patch
dpkg-source: info: applying ppc64le/third_party/0002-third_party-lss-kernel-structs.patch
dpkg-source: info: applying ppc64le/webrtc/Rtc_base-system-arch.h-PPC.patch
dpkg-source: info: applying ppc64le/crashpad/0002-Include-cstddef-to-fix-build.patch
dpkg-source: info: applying ppc64le/third_party/0004-third_party-crashpad-port-curl-transport-ppc64.patch
dpkg-source: info: applying ppc64le/workarounds/HACK-third_party-libvpx-use-generic-gnu.patch
dpkg-source: info: applying ppc64le/workarounds/HACK-debian-clang-disable-skia-musttail.patch
dpkg-source: info: applying ppc64le/libaom/0001-Add-ppc64-target-to-libaom.patch
dpkg-source: info: applying ppc64le/libaom/0001-Add-pregenerated-config-for-libaom-on-ppc64.patch
dpkg-source: info: applying ppc64le/third_party/0002-third_party-libvpx-Remove-bad-ppc64-config.patch
dpkg-source: info: applying ppc64le/third_party/0002-third-party-boringssl-add-generated-files.patch
dpkg-source: info: applying ppc64le/third_party/0003-third_party-libvpx-Add-ppc64-generated-config.patch
dpkg-source: info: applying ppc64le/third_party/0003-third_party-ffmpeg-Add-ppc64-generated-config.patch
dpkg-source: info: applying ppc64le/third_party/0004-third_party-libvpx-work-around-ambiguous-vsx.patch
dpkg-source: info: applying ppc64le/third_party/skia-vsx-instructions.patch
dpkg-source: info: applying ppc64le/ffmpeg/0001-Add-support-for-ppc64.patch
dpkg-source: info: applying ppc64le/breakpad/0001-Implement-support-for-ppc64-on-Linux.patch
dpkg-source: info: applying ppc64le/crashpad/0001-Implement-support-for-PPC64-on-Linux.patch
dpkg-source: info: applying ppc64le/database/0001-Properly-detect-little-endian-PPC64-systems.patch
dpkg-source: info: applying ppc64le/third_party/0001-Force-baseline-POWER8-AltiVec-VSX-CPU-features-when-.patch
dpkg-source: info: applying ppc64le/fixes/fix-breakpad-compile.patch
dpkg-source: info: applying ppc64le/fixes/fix-partition-alloc-compile.patch
dpkg-source: info: applying ppc64le/v8/0002-Add-ppc64-trap-instructions.patch
dpkg-source: info: applying ppc64le/third_party/0001-Fix-highway-ppc-hwcap.patch
dpkg-source: info: applying ppc64le/third_party/0001-Add-PPC64-support-for-libdav1d.patch
dpkg-source: info: applying ppc64le/third_party/0001-Fix-libdav1d-compilation-on-clang-ppc.patch
dpkg-source: info: applying ppc64le/sandbox/fix-ppc64-linux-syscalls-headers.patch
dpkg-source: info: applying ppc64le/third_party/0003-thirdparty-fix-dav1d-gn.patch
dpkg-source: info: applying ppc64le/third_party/use-sysconf-page-size-on-ppc64.patch
dpkg-source: info: applying ppc64le/third_party/dawn-fix-typos.patch
dpkg-source: info: applying ppc64le/third_party/dawn-fix-ppc64le-detection.patch
dpkg-source: info: applying ppc64le/fixes/fix-unknown-warning-option-messages.diff
dpkg-source: info: applying ppc64le/third_party/0002-Highway-disable-128-bit-vsx.patch
dpkg-source: info: applying ungoogled/core/ungoogled-chromium/disable-web-environment-integrity.patch

It seems different now: I do not see the convert folder anymore

iskunk commented 11 months ago

@Ark74: Thank you for bringing this to my attention. I was able to reproduce that error using QEMU.

It took some digging, but I was able to track down the issue to this bit in build/config/BUILDCONFIG.gn, around line 141:

  # Set to true when compiling with the Clang compiler.
  is_clang = current_os != "linux" ||
             (current_cpu != "s390x" && current_cpu != "s390" &&
              current_cpu != "ppc64" && current_cpu != "ppc" &&
              current_cpu != "mips" && current_cpu != "mips64" &&
              current_cpu != "riscv64")

The GN logic appears to assume that if you're building for any of those architectures, then you're not using Clang---which is clearly incorrect in the case of ppc64el. Please try removing/disabling the current_cpu != "ppc64" bit above, and give the build another go with use_thin_lto=true.

What errors were you seeing on arm64 / armhf?

Also, some clarifications on LTO:

@raphael10-collab: If you're not already comfortable with handling and building Debian source packages, then there is not much I can do to get you through it. Using the conversion framework I've put together requires experience in that area.

Could you talk about why you wish to build this yourself, instead of using pre-built packages? If you have concerns about the integrity of the build, then there may be an alternative way of addressing that.

Ark74 commented 11 months ago

@iskunk you are right, removing current_cpu != "ppc64" does pass the initial error on ppc64el, there are some hours ahead to confirm build, hopefully it will pass.

arm64/hf didn't show some clear error output, just failing randomly, I'm using a couple of RPi4 8GB RAM, but even if you get access to some AWS ARM metal servers, they also failed on my first tests.

I'll run some new tests, and comment on the output, regards.

Ark74 commented 11 months ago

@iskunk I can confirm now that ppc64el passes with that change.

I'm not sure if the v4l2 issues comes from trisquel or plain ubuntu, last time I succeed using this patch and disabling thin-lto.

iskunk commented 11 months ago

Yes, the v4l2 issue is due to Ubuntu jammy's kernel not being new enough. (Lunar and later are good.) That patch you linked is the right idea, but you should use these settings:

ifeq (arm64,$(DEB_HOST_ARCH))
defines+=host_cpu=\"arm64\" use_vaapi=true
endif
ifeq (armhf,$(DEB_HOST_ARCH))
defines+=host_cpu=\"arm\" use_vaapi=false arm_use_neon=true
endif

This is what Debian uses in their bullseye build. See this bug for background.

That html_to_wrapper.py error in the armhf build, however, that I have not seen before. I've successfully performed a full armhf build on jammy via QEMU, so I am not able to reproduce it. Could you try to see what's going on there? All I can say to help is that the third_party/node/linux/node-linux-x64/bin/node binary comes from this line in the rules file.

One thing I forgot to mention earlier: If you can, try to wrap your builds with env time --verbose. That will give you some useful statistics at the end, most notably the "Maximum resident set size." Knowing how much RAM a build requires, especially with ThinLTO vs. without, is critical on armhf. (My QEMU build yielded a max RSS of 2.4 GB without ThinLTO, so I think your RPis should be able to handle it with.)

Ark74 commented 11 months ago

Yes, the v4l2 issue is due to Ubuntu jammy's kernel not being new enough

I guess linux-hwe-6.2 could work, but that would require a non-standard dependency, so yeah, I guess it's better to disable it for now.

I've updated the logs for arm on a trisquel env, seems like the aws tripped somehow. I'll rebuild the sources with the suggested proposal and give it another round.

Update: Here it is the AWS build log, success for arm64, I guess this proves the point so this could be proven to actually work for jammy, I'll continue testing on the rpi4, once I have room.

Ark74 commented 11 months ago

@iskunk news on the armhf build on the rpi4. Well, it failed :disappointed:, but just for the hardware limitation.

I guess the important part is:

[57730/57749] CXX obj/chrome/browser/browser/download_target_determiner.o
[57731/57749] CXX obj/chrome/browser/browser/chrome_download_manager_delegate.o
[57732/57749] AR obj/chrome/browser/libbrowser.a
[57733/57749] LINK ./v8_context_snapshot_generator
FAILED: v8_context_snapshot_generator 
"python3" "../../build/toolchain/gcc_link_wrapper.py" --output="./v8_context_snapshot_generator" -- clang++-15 -fuse-ld=lld-15 -Wl,--fatal-warnings -Wl,--build-id=sha1 -fPIC -Wl,-z,noexecstack -Wl,-z,relro -Wl,-z,now -Wl,--icf=all -Wl,--color-diagnostics -Wl,-mllvm,-instcombine-lower-dbg-declare=0 -flto=thin -Wl,--thinlto-jobs=all -Wl,--thinlto-cache-dir=thinlto-cache -Wl,--thinlto-cache-policy=cache_size=10\%:cache_size_bytes=40g:cache_size_files=100000 -Wl,-mllvm,-import-instr-limit=30 -fwhole-program-vtables -march=armv7-a -Wl,--undefined-version -Wl,--no-call-graph-profile-sort --target=arm-linux-gnueabihf -no-canonical-prefixes -Wl,-O2 -Wl,--gc-sections -rdynamic -Wl,-z,defs -Wl,--as-needed -Wl,--lto-O0 -pie -Wl,--disable-new-dtags -Wl,--icf=none -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now -Wl,--stats -o "./v8_context_snapshot_generator" -Wl,--start-group @"./v8_context_snapshot_generator.rsp"  -Wl,--end-group  -latomic -ldl -lpthread -lrt -lgmodule-2.0 -lgobject-2.0 -lgthread-2.0 -lglib-2.0 -ljsoncpp -lnss3 -lnssutil3 -lsmime3 -lplds4 -lplc4 -lnspr4 -latk-1.0 -latk-bridge-2.0 -lcups -lz -ldouble-conversion -levent -lresolv -lgio-2.0 -lexpat -lpng16 -lwebpdemux -lwebpmux -lwebp -lfreetype -ljpeg -lfontconfig -ldrm -lopus -lopenh264 -lm -lX11 -lXcomposite -lXdamage -lXext -lXfixes -lXrender -lXrandr -lXtst -lgbm -lEGL -lxcb -lxkbcommon -lXi -lpci -l:libffi_pic.a -lXNVCtrl -ldbus-1 -lpangocairo-1.0 -lpango-1.0 -lharfbuzz -lcairo -lasound -lpulse -lFLAC -lsnappy -latspi -lxml2 -lminizip -lxslt -lwoff2dec -lbrotlidec 
ld.lld-15: error: failed to open ./v8_context_snapshot_generator: Cannot allocate memory
clang++-15: error: linker command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed.
make[1]: *** [debian/rules:161: override_dh_auto_build-arch] Error 1
make[1]: Leaving directory '/<<PKGBUILDDIR>>'
make: *** [debian/rules:123: binary] Error 2
dpkg-buildpackage: error: debian/rules binary subprocess returned exit status 2

About env time --verbose, here the output:

Command exited with non-zero status 2
        Command being timed: "sbuild --no-run-lintian -v -j 4 -A --dist aramo --arch armhf --build-dep-resolver=aptitude ungoogled-chromium_117.0.5938.149-1~xtradeb1+11.0trisquel1.dsc --extra-repository=deb http://ports.ubuntu.com/ubuntu-ports/ jammy main"
        User time (seconds): 860058.18
        System time (seconds): 31953.59
        Percent of CPU this job got: 395%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 62:43:40
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 3598344
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 16621
        Minor (reclaiming a frame) page faults: 1662971621
        Voluntary context switches: 2424058
        Involuntary context switches: 56859450
        Swaps: 0
        File system inputs: 5405688
        File system outputs: 31998432
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 2

This rpi has a 8GB swap file, I guess I could easily double that, but I'm not sure if that would work / be enough, any suggestion? I guess that this still proves it works, just need more resources than tested on this SBC.

Regards.

iskunk commented 11 months ago

@Ark74: I didn't expect the armhf build max RSS to top out at under 4 GB RAM. Can armhf programs not address more than that, like (32-bit) x86 with PAE?

I'm currently running an armhf build in QEMU with use_thin_lto=true. That will not be limited by available RAM, but if the limitation is architectural, then I should run into the same failure as you did.

In which case, the only way to build a ThinLTO-optimized Chromium for armhf would be to cross-compile. I don't believe the source package is set up to allow cross-compilation...

@raphael10-collab: You might want to try ungoogled-chromium in the XtraDeb applications PPA. The source has been converted, and there are binary builds there too if you want. The downside is that the version is a bit behind---@jhonny-oliveira and I are still working on the update infrastructure.

iskunk commented 11 months ago

An update: I've confirmed that the final link, on armhf with ThinLTO enabled, is impossible.

The build failed similarly at the v8_context_snapshot_generator link. I was able to get past that by running the link command directly, but then the final link of chrome fails even then:

build@xtradeb-jammy-armhf-ubuntu-con:~/chromium/chromium-118.0.5993.70/out/Release$ time -p "python3" "../../build/toolchain/gcc_link_wrapper.py" --output="./chrome" -- clang++-15 -Wl,--version-script=../../build/linux/chrome.map -fuse-ld=lld-15 -Wl,--fatal-warnings -Wl,--build-id=sha1 -fPIC -Wl,-z,noexecstack -Wl,-z,relro -Wl,-z,now -Wl,--icf=all -Wl,--color-diagnostics -Wl,-mllvm,-instcombine-lower-dbg-declare=0 -flto=thin -Wl,--thinlto-jobs=all -Wl,--thinlto-cache-dir=thinlto-cache -Wl,--thinlto-cache-policy=cache_size=10\%:cache_size_bytes=40g:cache_size_files=100000 -Wl,-mllvm,-import-instr-limit=30 -fwhole-program-vtables -march=armv7-a -Wl,--undefined-version -Wl,--no-call-graph-profile-sort --target=arm-linux-gnueabihf -no-canonical-prefixes -Wl,-O2 -Wl,--gc-sections -rdynamic -Wl,-z,defs -Wl,--as-needed -pie -Wl,--disable-new-dtags -Wl,--lto-O2 -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now -Wl,--stats -o "./chrome" -Wl,--start-group @"./chrome.rsp"  -Wl,--end-group  -latomic -ldl -lpthread -lrt -lgmodule-2.0 -lgobject-2.0 -lgthread-2.0 -lglib-2.0 -ljsoncpp -lnss3 -lnssutil3 -lsmime3 -lplds4 -lplc4 -lnspr4 -latk-1.0 -latk-bridge-2.0 -lcups -lgio-2.0 -lfontconfig -ldbus-1 -lz -ldouble-conversion -levent -latspi -lresolv -lexpat -lpng16 -lwebpdemux -lwebpmux -lwebp -lfreetype -ljpeg -lopus -lopenh264 -lm -lX11 -lXcomposite -lXdamage -lXext -lXfixes -lXrender -lXrandr -lXtst -lgbm -lEGL -ldrm -lxcb -lxkbcommon -lXi -lpci -l:libffi_pic.a -lXNVCtrl -lpangocairo-1.0 -lpango-1.0 -lharfbuzz -lcairo -lasound -lpulse -lFLAC -lsnappy -lxml2 -lminizip -lxslt -lwoff2dec -lbrotlidec -lxshmfence -lopenjp2 -llcms2 -lpng
LLVM ERROR: out of memory
Allocation failed
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.
Stack dump:
0.  Program arguments: /usr/bin/ld.lld-15 @/tmp/response-e65857.txt
clang++-15: error: unable to execute command: Aborted (core dumped)
clang++-15: error: linker command failed due to signal (use -v to see invocation)
real 295.54
user 279.98
sys 13.45

My hope that there was some way a 32-bit program could access more than 4 GB memory was for naught :-(

So that confirms, then, that building Chromium with ThinLTO is not possible on an armhf build host, and probably any 32-bit host as well (so also x86). Which is a bummer, because those platforms need all the speed-ups they can get.

For the record, here are the stats on the bulk of my (armhf via QEMU) build. @Ark74's little RPi beat the snot out of my 8-CPU VM:

Command exited with non-zero status 2
    Command being timed: "ionice -c 3 nice -99 dpkg-buildpackage -b"
    User time (seconds): 3631058.76
    System time (seconds): 30246.36
    Percent of CPU this job got: 2%
    Elapsed (wall clock) time (h:mm:ss or m:ss): 127:29:08
    Average shared text size (kbytes): 0
    Average unshared data size (kbytes): 0
    Average stack size (kbytes): 0
    Average total size (kbytes): 0
    Maximum resident set size (kbytes): 3594472
    Average resident set size (kbytes): 0
    Major (requiring I/O) page faults: 37749
    Minor (reclaiming a frame) page faults: 2104661801
    Voluntary context switches: 7174060
    Involuntary context switches: 89119022
    Swaps: 0
    File system inputs: 216
    File system outputs: 13338312
    Socket messages sent: 0
    Socket messages received: 0
    Signals delivered: 0
    Page size (bytes): 4096
    Exit status: 2
Ark74 commented 11 months ago

@iskunk I wonder if linux-image-generic-lpae could make any difference on the memory usage.

I'm running a test, we'll find out later in the week.

Update: No change, so yeah seems like armhf should have thin-lto disabled.