ungoogled-software / ungoogled-chromium

Google Chromium, sans integration with Google
BSD 3-Clause "New" or "Revised" License
20.21k stars 818 forks source link

A bit help in building Ungoogled-Chromium #1256

Closed TCH68k closed 2 years ago

TCH68k commented 3 years ago

Hello,

I would like to build Ungoogled-Chromium, but i stucked, because there is no description how i connect the sources here to the official Chromium sources.

I've read the building.md here: https://github.com/Eloston/ungoogled-chromium/blob/master/docs/building.md It says it will assume that i can build Chromium on my own. I can, but what after? Or before? I do not understand, i should put Chromium sources into a directory inside Ungoogled-Chromium or copy the files of Ungoogled-Chromium into the directory of Chromium? The description does not cover this part...

TCH68k commented 3 years ago

Only a few percentage left, but here is another one:

[8630/9283] CXX obj/v8/v8_base/js-locale.o
FAILED: obj/v8/v8_base/js-locale.o
../../third_party/llvm-build/Release+Asserts/bin/clang++ -MMD -MF obj/v8/v8_base/js-locale.o.d -DUSE_UDEV -DUSE_AURA=1 -DUSE_GLIB=1 -DUSE_NSS_CERTS=1 -DUSE_X11=1 -DCHROMIUM_BUILD -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -DCR_CLANG_REVISION=\"351477-1\" -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D_LIBCPP_DISABLE_VISIBILITY_ANNOTATIONS -D_LIBCXXABI_DISABLE_VISIBILITY_ANNOTATIONS -DCR_LIBCXX_REVISION=349080 -DCR_LIBCXXABI_REVISION=347903 -D_LIBCPP_ENABLE_NODISCARD -D_LIBCPP_ABI_UNSTABLE -DNDEBUG -DNVALGRIND -DDYNAMIC_ANNOTATIONS_ENABLED=0 -DV8_TYPED_ARRAY_MAX_SIZE_IN_HEAP=64 -DENABLE_MINOR_MC -DV8_DEPRECATION_WARNINGS -DV8_INTL_SUPPORT -DV8_USE_SNAPSHOT -DV8_USE_EXTERNAL_STARTUP_DATA -DV8_CONCURRENT_MARKING -DV8_EMBEDDED_BUILTINS -DV8_TARGET_ARCH_X64 -DDISABLE_UNTRUSTED_CODE_MITIGATIONS -DU_USING_ICU_NAMESPACE=0 -DU_ENABLE_DYLOAD=0 -DUSE_CHROMIUM_ICU=1 -DU_STATIC_IMPLEMENTATION -DICU_UTIL_DATA_IMPL=ICU_UTIL_DATA_FILE -DUCHAR_TYPE=uint16_t -I../.. -Igen -I../../v8 -Igen/v8 -I../../third_party/icu/source/common -I../../third_party/icu/source/i18n -I../../v8/include -fno-strict-aliasing --param=ssp-buffer-size=4 -fstack-protector -Wno-builtin-macro-redefined -D__DATE__= -D__TIME__= -D__TIMESTAMP__= -funwind-tables -fPIC -B../../third_party/binutils/Linux_x64/Release/bin -pthread -fcolor-diagnostics -fmerge-all-constants -Xclang -mllvm -Xclang -instcombine-lower-dbg-declare=0 -no-canonical-prefixes -fcomplete-member-pointers -m64 -march=x86-64 -Wall -Wextra -Wimplicit-fallthrough -Wthread-safety -Wno-missing-field-initializers -Wno-unused-parameter -Wno-c++11-narrowing -Wno-unneeded-internal-declaration -Wno-undefined-var-template -Wno-ignored-pragma-optimize -fno-omit-frame-pointer -g0 -fvisibility=hidden -Xclang -add-plugin -Xclang find-bad-constructs -Xclang -plugin-arg-find-bad-constructs -Xclang check-ipc -Wheader-hygiene -Wstring-conversion -Wtautological-overlap-compare -Wexit-time-destructors -Wmissing-field-initializers -Winconsistent-missing-override -Wunreachable-code -Wshorten-64-to-32 -O3 -fno-ident -fdata-sections -ffunction-sections -std=c++14 -fno-exceptions -fno-rtti -nostdinc++ -isystem../../buildtools/third_party/libc++/trunk/include -isystem../../buildtools/third_party/libc++abi/trunk/include -fvisibility-inlines-hidden -c ../../v8/src/objects/js-locale.cc -o obj/v8/v8_base/js-locale.o
../../v8/src/objects/js-locale.cc:34:51: error: unknown type name 'StringPiece'; did you mean 'icu_63::StringPiece'?
getUnicodeKeywordValue(const icu::Locale& locale, StringPiece keywordName, ByteSink& sink, UErrorCode& status)
                                                  ^~~~~~~~~~~
                                                  icu_63::StringPiece
../../third_party/icu/source/common/unicode/stringpiece.h:54:20: note: 'icu_63::StringPiece' declared here
class U_COMMON_API StringPiece : public UMemory {
                   ^
../../v8/src/objects/js-locale.cc:34:76: error: unknown type name 'ByteSink'; did you mean 'icu_63::ByteSink'?
getUnicodeKeywordValue(const icu::Locale& locale, StringPiece keywordName, ByteSink& sink, UErrorCode& status)
                                                                           ^~~~~~~~
                                                                           icu_63::ByteSink
../../third_party/icu/source/common/unicode/bytestream.h:50:20: note: 'icu_63::ByteSink' declared here
class U_COMMON_API ByteSink : public UMemory {
                   ^
../../v8/src/objects/js-locale.cc:69:51: error: unknown type name 'StringPiece'; did you mean 'icu_63::StringPiece'?
getUnicodeKeywordValue(const icu::Locale& locale, StringPiece keywordName, UErrorCode& status)
                                                  ^~~~~~~~~~~
                                                  icu_63::StringPiece
../../third_party/icu/source/common/unicode/stringpiece.h:54:20: note: 'icu_63::StringPiece' declared here
class U_COMMON_API StringPiece : public UMemory {
                   ^
../../v8/src/objects/js-locale.cc:72:5: error: no template named 'StringByteSink'; did you mean 'icu_63::StringByteSink'?
    StringByteSink<StringClass> sink(&result);
    ^~~~~~~~~~~~~~
    icu_63::StringByteSink
../../third_party/icu/source/common/unicode/bytestream.h:232:7: note: 'icu_63::StringByteSink' declared here
class StringByteSink : public ByteSink {
      ^
4 errors generated.

This is caused by the icu57.patch from the Debian patchset. I really don't know how to fix this properly, so i just accepted what the compiler has been suggested.

TCH68k commented 3 years ago

This time linking has failed, again on the event handling:

[623/632] LINK ./v8_context_snapshot_generator
FAILED: v8_context_snapshot_generator
python "../../build/toolchain/gcc_link_wrapper.py" --output="./v8_context_snapshot_generator" -- ../../third_party/llvm-build/Release+Asserts/bin/clang++ -Wl,--fatal-warnings -fPIC -Wl,-z,noexecstack -Wl,-z,now -Wl,-z,relro -Wl,-z,defs -Wl,--no-as-needed -fuse-ld=lld -Wl,--icf=all -Wl,--color-diagnostics -m64 -Wl,-O2 -Wl,--gc-sections -rdynamic -nostdlib++ -pie -Wl,-rpath-link=. -Wl,--disable-new-dtags -Wl,--icf=none -o "./v8_context_snapshot_generator" -Wl,--start-group @"./v8_context_snapshot_generator.rsp"  -Wl,--end-group   -ldl -lpthread -lrt -lgmodule-2.0 -lgobject-2.0 -lgthread-2.0 -lglib-2.0 -lnss3 -lnssutil3 -lsmime3 -lplds4 -lplc4 -lnspr4 -latomic -lresolv -lgio-2.0 -lX11 -lX11-xcb -lxcb -lXcomposite -lXcursor -lXdamage -lXext -lXfixes -lXi -lXrender -lXtst -lexpat -luuid -lXrandr -lpangocairo-1.0 -lpango-1.0 -lcairo -lXss -lasound -lm -lz -lpci -ldbus-1
ld.lld: error: undefined symbol: event_get_fd
>>> referenced by message_pump_libevent.cc
>>>               base/message_pump_libevent.o:(base::MessagePumpLibevent::WatchFileDescriptor(int, bool, int, base::MessagePumpLibevent::FdWatchController*, base::WatchableIOMessagePumpPosix::FdWatcher*)) in archive obj/base/libbase.a
clang: error: linker command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed.
TCH68k commented 3 years ago

I managed to bypass that, by adding the -levent flag manually to the linker flags in tools/v8_context_snapshot/BUILD.gn. Now only the chrome binary's linking is remaining, but that crashes too:

[11/11] LINK ./chrome
FAILED: chrome
python "../../build/toolchain/gcc_link_wrapper.py" --output="./chrome" -- ../../third_party/llvm-build/Release+Asserts/bin/clang++ -Wl,--version-script=../../build/linux/chrome.map -Wl,--fatal-warnings -fPIC -Wl,-z,noexecstack -Wl,-z,now -Wl,-z,relro -Wl,-z,defs -Wl,--no-as-needed -fuse-ld=lld -Wl,--icf=all -Wl,--color-diagnostics -m64 -Wl,-O2 -Wl,--gc-sections -rdynamic -nostdlib++ -pie -Wl,-rpath-link=. -Wl,--disable-new-dtags -o "./chrome" -Wl,--start-group @"./chrome.rsp"  -Wl,--end-group   -ldl -lpthread -lrt -lX11 -lX11-xcb -lxcb -lXcomposite -lXcursor -lXdamage -lXext -lXfixes -lXi -lXrender -lXtst -lgmodule-2.0 -lgobject-2.0 -lgthread-2.0 -lglib-2.0 -lnss3 -lnssutil3 -lsmime3 -lplds4 -lplc4 -lnspr4 -lcups -ldbus-1 -latomic -lXss -lexpat -luuid -lXrandr -lresolv -lgio-2.0 -lasound -lm -lz -lpci -lpangocairo-1.0 -lpango-1.0 -lcairo -latk-1.0 -latspi -lgtk-x11-2.0 -lgdk-x11-2.0 -lgdk_pixbuf-2.0 -lpangoft2-1.0 -lfontconfig -lfreetype -lopenjp2
ld.lld: error: undefined symbol: event_get_fd
>>> referenced by message_pump_libevent.cc
>>>               base/message_pump_libevent.o:(base::MessagePumpLibevent::WatchFileDescriptor(int, bool, int, base::MessagePumpLibevent::FdWatchController*, base::WatchableIOMessagePumpPosix::FdWatcher*)) in archive obj/base/libbase.a

ld.lld: error: undefined symbol: u_strFromUTF8_57
>>> referenced by string_conversion.cc
>>>               client/string_conversion.o:(google_breakpad::UTF8ToUTF16Char(char const*, int, unsigned short*)) in archive obj/third_party/breakpad/libclient.a

ld.lld: error: undefined symbol: OneClickSigninSyncStarter::OneClickSigninSyncStarter(Profile*, Browser*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, signin_metrics::AccessPoint, signin_metrics::Reason, OneClickSigninSyncStarter::ProfileMode, base::RepeatingCallback<void (OneClickSigninSyncStarter::SyncSetupResult)>)
>>> referenced by inline_login_handler_impl.cc
>>>               ui/inline_login_handler_impl.o:(InlineSigninHelper::CreateSyncStarter(Browser*, GURL const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, OneClickSigninSyncStarter::ProfileMode)) in archive obj/chrome/browser/ui/libui.a

ld.lld: error: undefined symbol: OneClickSigninLinksDelegateImpl::OneClickSigninLinksDelegateImpl(Browser*)
>>> referenced by browser_view.cc
>>>               ui/browser_view.o:(BrowserView::ShowOneClickSigninConfirmation(std::__1::basic_string<unsigned short, base::string16_internals::string16_char_traits, std::__1::allocator<unsigned short> > const&, base::OnceCallback<void (bool)>)) in archive obj/chrome/browser/ui/libui.a

ld.lld: error: undefined symbol: OneClickSigninDialogView::ShowDialog(std::__1::basic_string<unsigned short, base::string16_internals::string16_char_traits, std::__1::allocator<unsigned short> > const&, std::__1::unique_ptr<OneClickSigninLinksDelegate, std::__1::default_delete<OneClickSigninLinksDelegate> >, aura::Window*, base::OnceCallback<void (bool)>)
>>> referenced by browser_view.cc
>>>               ui/browser_view.o:(BrowserView::ShowOneClickSigninConfirmation(std::__1::basic_string<unsigned short, base::string16_internals::string16_char_traits, std::__1::allocator<unsigned short> > const&, base::OnceCallback<void (bool)>)) in archive obj/chrome/browser/ui/libui.a
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Now which flags i should add the linker and where i have to do it?

TCH68k commented 3 years ago

I've found the linker flags in chrome/BUILD.gn and i've added -levent to it, eliminating the first error and -licuuc, eliminating the second. However, the rest are seems to be locally defined.

TCH68k commented 3 years ago

Solved by setting enable_one_click_signin to false. 1000 objects needs to be redone...

TCH68k commented 3 years ago

Success: http://oscomp.hu/depot/ungoogled-chromium-gtk2.png

Thank you for your help, i could have not done it without you. Tomorrow i'll try to build it with the flags.gn and not by tweaking the separate files.

The final question is, how to collect all needed files from the out/Release directory? objdump -p ./chrome | grep NEEDED and copy them by hand, or there is some mechanism for this?

wchen342 commented 3 years ago

Glad to see it finally worked for you!

About the files, AFAIK there is no automatic way. I think you can just find a Debian package and inspect what files are in it. If that is not possible than maybe have a look at the Arch PKGBUILD for the package() section or the %files section in Fedora .spec file.

TCH68k commented 3 years ago

Well, no problem, i've got it, it's just the .pak, .info, .so, .bin and .dat files were needed aside from the chrome binary. I've tried to build it with the flags.gn though and the resulting binary was crashed by "illegal instruction". Dunno why. If i build like this, everything works. The - now fully working - script:

#!/bin/sh
cd "$HOME"
mkdir kocsog_gugli 2>/dev/null
cd kocsog_gugli

echo
echo "======================"
echo "Getting depot tools..."
echo "======================"
echo
git clone https://chromium.googlesource.com/chromium/tools/depot_tools.git
export PATH=$PATH:"$HOME"/kocsog_gugli/depot_tools

echo
echo "=========================="
echo "Getting Chromium source..."
echo "=========================="
echo
mkdir chromium 2>/dev/null
cd chromium/
fetch --nohooks chromium
free > /dev/null && sync > /dev/null && echo 3 > /proc/sys/vm/drop_caches && free > /dev/null

echo
echo "===================="
echo "Checkout and sync..."
echo "===================="
echo
cd src
git checkout 73.0.3683.75
gclient sync -D
free > /dev/null && sync > /dev/null && echo 3 > /proc/sys/vm/drop_caches && free > /dev/null

# Use the "fontconfig.patch" from UGC and not from Debian. The rest of the two patchset can be merged.
echo
echo "==========="
echo "Patching..."
echo "==========="
echo
echo "Getting Debian patchset..."
wget http://deb.debian.org/debian/pool/main/c/chromium/chromium_73.0.3683.75-1~deb9u1.debian.tar.xz
tar xf chromium_73.0.3683.75-1~deb9u1.debian.tar.xz
rm chromium_73.0.3683.75-1~deb9u1.debian.tar.xz
rm -rf debian/patches/arm
find debian/patches/ -name '*.patch' -exec mv {} ./ \;
rm -rf debian
echo "Getting Ungoogled-Chromium patchset..."
git clone https://github.com/Eloston/ungoogled-chromium/
cd ungoogled-chromium/
git checkout 73.0.3683.103-2
cd ..
find ungoogled-chromium/patches/ -name '*.patch' -exec mv {} ./ \;
rm -rf ungoogled-chromium
echo "Applying patches..."
for PATCH in *.patch
do
    echo "****************"
    echo "* ""$PATCH"
    echo "****************"
    patch -p1 -f < "$PATCH"
    echo "****************"
    echo
done
rm *.patch
# Some files are left in an uncompilable state.
echo "Some hand patching..."
# Designated initializers patch is erroneous with the bundled CLang 9.
sed -i.bak 's/    Settings(bool randomise):randomised_sampling_enabled(randomise) {}//g' base/task/sequence_manager/sequence_manager.h
# Patch a typecast error.
sed -i.bak 's/buffer = host_alt/buffer = \(UChar \*\)host_alt/g' components/url_formatter/idn_spoof_checker.cc
# Patch a mispatched file's macro.
sed -i.bak -n '1h;1!H;${g;s/#endif\n#endif/#endif/;p;}' chrome/browser/ui/chrome_pages.cc
# Patch some missing ICU scopes and removes redefinition.
sed -i.bak -n 's/StringPiece/icu_63::StringPiece/g; s/StringByteSink/SBPLACEHOLDER/g; s/ByteSink/icu_63::ByteSink/g; s/SBPLACEHOLDER/icu_63::StringByteSink/g; 1h;1!H;${g;s/Adapted from icu\(.*\)Adapted from icu/Adapted from icu/;p;}' v8/src/objects/js-locale.cc
# Add libevent manually to linker flags in v8_context_snapshot.
sed -i.bak 's/      ldflags = \[ "-Wl,--icf=none" \]/      ldflags = \[ "-Wl,--icf=none", "-levent" \]/g' tools/v8_context_snapshot/BUILD.gn
# Add libevent and libicuuc to linker flags in chrome.
sed -i.bak 's/        ldflags = \[\]/        ldflags = \[ "-levent", "-licuuc" \]/g' chrome/BUILD.gn
# Safe browsing patch is erroneous with chrome 73.0.
sed -i.bak '/^static_library("safe_browsing") {/i sources = []\n deps = []' chrome/browser/safe_browsing/BUILD.gn
# flags.gn is ENTIRELY ignored by the buildsystem, but even if we force it's content into the args.gn, the resulting binary will crash with "illegal instruction". Therefore we have to tweak the settings by hand.
echo "Tweaking settings..."
# Disable gtk3, use gtk2.
sed -i.bak 's/  gtk_version = 3/  gtk_version = 2/g' build/config/linux/gtk/gtk.gni
# Disable pulseaudio, use alsa.
sed -i.bak 's/      use_pulseaudio = true/      use_pulseaudio = false/g' media/media_options.gni
# Disable sysroot.
sed -i.bak 's/  use_sysroot = current_cpu == "x86" || current_cpu == "x64" ||/  use_sysroot = false/g; s/                current_cpu == "arm" || current_cpu == "arm64" ||//g; s/                current_cpu == "mipsel" || current_cpu == "mips64el"//g' build/config/sysroot.gni
# Disable gnome-keyring.
sed -i.bak 's/  use_gnome_keyring = is_desktop_linux && use_glib/  use_gnome_keyring = false/g' components/os_crypt/features.gni
# Force disabling "closure_compile".
sed -i.bak 's/  closure_compile = is_chromeos || is_linux || is_android/  closure_compile = false/g' ui/webui/webui_features.gni
# Disable nacl by hand.
sed -i.bak 's/  enable_nacl =/  enable_nacl = false/g; s/      checkout_nacl && !is_ios && !is_android && !is_fuchsia &&//g; s/      !is_chromecast && current_cpu != "mipsel" && current_cpu != "mips64el" &&//g; s/      !(is_linux && target_cpu == "arm64") && !(is_win && host_os != "win")//g' components/nacl/features.gni
# Disable safe browsing.
sed -i.bak 's/    safe_browsing_mode = 1/    safe_browsing_mode = 0/g' build/config/features.gni
# Disable halt on warnings.
sed -i.bak 's/  treat_warnings_as_errors = true/  treat_warnings_as_errors = false/g' build/config/compiler/BUILD.gn
# Disable one click signin.
sed -i.bak 's/  enable_one_click_signin =/  enable_one_click_signin = false/g; s/      is_win || is_mac || (is_linux && !is_chromeos && !is_chromecast)//g' chrome/common/features.gni

echo
echo "============="
echo "Generating..."
echo "============="
echo
export PKG_CONFIG_PATH="$PKG_CONFIG_PATH":/usr/lib/x86_64-linux-gnu/pkgconfig/
gn gen out/Release "--args=is_debug=false"
gn args --list out/Release/ > ninja.settings
free > /dev/null && sync > /dev/null && echo 3 > /proc/sys/vm/drop_caches && free > /dev/null

echo
echo "==========="
echo "Building..."
echo "==========="
echo
autoninja -C out/Release chrome
free > /dev/null && sync > /dev/null && echo 3 > /proc/sys/vm/drop_caches && free > /dev/null

echo
echo "============"
echo "Packaging..."
echo "============"
echo
cd out/Release
mkdir ungoogled-chromium-devuan2-gtk2-portable
cp -R locales chrome *.pak *.info *.so *.bin *.dat ungoogled-chromium-devuan2-gtk2-portable/
tar -zcf ungoogled-chromium-devuan2-gtk2-portable.tar.gz ungoogled-chromium-devuan2-gtk2-portable
rm -rf ungoogled-chromium-devuan2-gtk2-portable
mv ungoogled-chromium-devuan2-gtk2-portable.tar.gz "$HOME"/
free > /dev/null && sync > /dev/null && echo 3 > /proc/sys/vm/drop_caches && free > /dev/null

if [ "$1" != "" ];
then
    echo
    echo "=============="
    echo "Cleaning up..."
    echo "=============="
    echo
    rm -rf "$HOME"/kocsog_gugli
    free > /dev/null && sync > /dev/null && echo 3 > /proc/sys/vm/drop_caches && free > /dev/null
fi
wchen342 commented 3 years ago

Well if you do not use the full set of flags then you are leaving certain functionalities open to Google, like remoting, so strictly speaking it is no longer "ungoogled", but at least you will have a working browser. I am not sure where the "illegal instruction" comes from but if you build it with is_debug=true and symbol_level=1 then it will probably give useful tracebacks.

TCH68k commented 3 years ago

I thought the ungoogling part is done by the patches... I've changed my script's generating part to this

echo
echo "============="
echo "Generating..."
echo "============="
echo
export PKG_CONFIG_PATH="$PKG_CONFIG_PATH":/usr/lib/x86_64-linux-gnu/pkgconfig/
wget https://raw.githubusercontent.com/Eloston/ungoogled-chromium/73.0.3683.103-2/flags.gn
echo 'gtk_version=2' >> flags.gn
echo 'use_gnome_keyring=false' >> flags.gn
echo 'use_pulseaudio=false' >> flags.gn
echo 'use_sysroot=false' >> flags.gn
sed -i.bak 's/is_debug=false/is_debug=true/g; s/symbol_level=0/symbol_level=1/g' flags.gn
gn gen out/Release "--args=`cat flags.gn | tr '\n' ' '`"
gn args --list out/Release/ > ninja.settings
free > /dev/null && sync > /dev/null && echo 3 > /proc/sys/vm/drop_caches && free > /dev/null

and i've removed the handmade tweakings from the patching part. After nearly 7 hours from now, the build will be complete. Can you please tell me, how to change the build parameters without forcing the buildsystem to rebuild the entire project? flags.gn is entirely ignored by the buildsystem. out/Release/args.gn is not, but if i run

gn gen out/Release "--args=`cat flags.gn | tr '\n' ' '`"

then it will force the

autoninja -C out/Release chrome

command to rebuild everything, right? Or not? Or i can simply edit out/Release/args.gn and then run autoninja and it will only rebuild what needs to be rebuild?

TCH68k commented 3 years ago

By setting is_debug=true and symbol_level=1 i got

ERROR at //build/config/BUILDCONFIG.gn:168:1: Assertion failed.
assert(!(is_debug && is_official_build), "Can't do official debug builds")
^-----
Can't do official debug builds                                                  

Of course i left the machine to work... Set is_official_build = false with new command:

sed -i.bak 's/is_debug=false/is_debug=true/g; s/symbol_level=0/symbol_level=1/g; s/is_official_build=true/is_official_build=false/g' flags.gn

(This also sets blink_symbol_level but that is actually needed for debug.) Trying to start build now. That's five hours...

TCH68k commented 3 years ago

It seems that whenever libevent is invoked, it will die. Strange, because i've already built it with flags.gn and the build finished, only the binary was faulted. How can i force -levent globally?

Error message:

[9193/35024] SOLINK ./libbase.so
FAILED: libbase.so libbase.so.TOC
python "../../build/toolchain/gcc_solink_wrapper.py" --readelf="readelf" --nm="nm" --sofile="./libbase.so" --tocfile="./libbase.so.TOC" --output="./libbase.so" -- ../../third_party/llvm-build/Release+Asserts/bin/clang++ -shared -fPIC -Wl,-z,noexecstack -Wl,-z,now -Wl,-z,relro -Wl,-z,defs -Wl,--no-as-needed -fuse-ld=lld -Wl,--icf=all -Wl,--color-diagnostics -m64 -rdynamic -nostdlib++ -Wl,-rpath=\$ORIGIN/. -Wl,-rpath-link=. -lnspr4 -o "./libbase.so" -Wl,-soname="libbase.so" @"./libbase.so.rsp"
ld.lld: error: undefined symbol: event_get_fd
>>> referenced by message_pump_libevent.cc:164 (../../base/message_loop/message_pump_libevent.cc:164)
>>>               obj/base/base/message_pump_libevent.o:(base::MessagePumpLibevent::WatchFileDescriptor(int, bool, int, base::MessagePumpLibevent::FdWatchController*, base::WatchableIOMessagePumpPosix::FdWatcher*))

ld.lld: error: undefined symbol: event_get_fd
>>> referenced by message_pump_libevent.cc:165 (../../base/message_loop/message_pump_libevent.cc:165)
>>>               obj/base/base/message_pump_libevent.o:(base::MessagePumpLibevent::WatchFileDescriptor(int, bool, int, base::MessagePumpLibevent::FdWatchController*, base::WatchableIOMessagePumpPosix::FdWatcher*))

ld.lld: error: undefined symbol: event_get_fd
>>> referenced by message_pump_libevent.cc:175 (../../base/message_loop/message_pump_libevent.cc:175)
>>>               obj/base/base/message_pump_libevent.o:(base::MessagePumpLibevent::WatchFileDescriptor(int, bool, int, base::MessagePumpLibevent::FdWatchController*, base::WatchableIOMessagePumpPosix::FdWatcher*))

ld.lld: error: undefined symbol: event_get_fd
>>> referenced by message_pump_libevent.cc:181 (../../base/message_loop/message_pump_libevent.cc:181)
>>>               obj/base/base/message_pump_libevent.o:(base::MessagePumpLibevent::WatchFileDescriptor(int, bool, int, base::MessagePumpLibevent::FdWatchController*, base::WatchableIOMessagePumpPosix::FdWatcher*))
clang: error: linker command failed with exit code 1 (use -v to see invocation)
wchen342 commented 3 years ago

By default GN reads args.gn in your build folder so whenever you want to change the flags just change that file by substituting its contents. Most of the time ninja will identify the files needed to be rebuilt correctly with rare exceptions.

The libevent error seems to be a problem with system libevent in old Debian patches, see https://github.com/Eloston/ungoogled-chromium/issues/266#issuecomment-337654506. ~Solution is to remove the part regarding~

-#include "base/third_party/libevent/event.h"
+#include <event.h>

~in debian/patches/system/event.patch so it will build with bundled libevent.~

EDIT: Probably better just remove that patch altogether.

TCH68k commented 3 years ago

I know that the event.patch changed this and i can just rm it after the copying, but did not the Debian Team put that patch into the set for a reason? Or i can ignore that, they just do not trust bundled libs?

wchen342 commented 3 years ago

I think the main reason for using system libs is to minimize the package size because ever static linking increases the final size. In some cases using system libs is to fix some problem but I think this is not the case here.

TCH68k commented 3 years ago

I see. Well i did remove the event.patch and now it's the ICU what it cannot find:

[9259/35024] SOLINK ./libcrash_key.so
FAILED: libcrash_key.so libcrash_key.so.TOC
python "../../build/toolchain/gcc_solink_wrapper.py" --readelf="readelf" --nm="nm" --sofile="./libcrash_key.so" --tocfile="./libcrash_key.so.TOC" --output="./libcrash_key.so" -- ../../third_party/llvm-build/Release+Asserts/bin/clang++ -shared -fPIC -Wl,-z,noexecstack -Wl,-z,now -Wl,-z,relro -Wl,-z,defs -Wl,--no-as-needed -fuse-ld=lld -Wl,--icf=all -Wl,--color-diagnostics -m64 -rdynamic -nostdlib++ -Wl,-rpath=\$ORIGIN/. -Wl,-rpath-link=. -o "./libcrash_key.so" -Wl,-soname="libcrash_key.so" @"./libcrash_key.so.rsp"
ld.lld: error: undefined symbol: u_strFromUTF8_57
>>> referenced by string_conversion.cc:44 (../../third_party/breakpad/breakpad/src/common/string_conversion.cc:44)
>>>               client/string_conversion.o:(google_breakpad::UTF8ToUTF16Char(char const*, int, unsigned short*)) in archive obj/third_party/breakpad/libclient.a

ld.lld: error: undefined symbol: u_strFromWCS_57
>>> referenced by string_conversion.cc:56 (../../third_party/breakpad/breakpad/src/common/string_conversion.cc:56)
>>>               client/string_conversion.o:(google_breakpad::UTF32ToUTF16Char(wchar_t, unsigned short*)) in archive obj/third_party/breakpad/libclient.a
clang: error: linker command failed with exit code 1 (use -v to see invocation)
[9268/35024] CC obj/third_party/sqlite/chromium_sqlite3/sqlite3_shim.o
ninja: build stopped: subcommand failed.

Is there a bundled ICU in the Chromium source? If yes, i think i should remove the ICU patches too...

wchen342 commented 3 years ago

Yes there is an icu bundled. Actually chromium is known to have problems with system icu in a lot of cases so no reason not to use the bundled one.

TCH68k commented 3 years ago

It seems, it does not matter if the bundled or the system ICU is used... The very same error happened. u_strFromUTF8_57 and u_strFromWCS_57 are part of libicu 5.7, what i have installed, but if the bundled ICU is used, then why does not it finds it? Maybe there is another flag which changes between the bundled and the system ICU? use_platform_icu_alternatives is false, so it should be the bundled isn't it?

wchen342 commented 3 years ago

I think what I did in the past when facing a similar problem was just remove the call to whatever function causes the problem. The error is in breakpad which is a bug reporting system to Google so you are not likely to use it anyway, so that way is simpler.

TCH68k commented 3 years ago

I did so, i commented those calls out. (Why the bug reporting has not been disabled if it's not used?) I've ran into another error:

[8539/21834] SOLINK ./libpublic.so
FAILED: libpublic.so libpublic.so.TOC
python "../../build/toolchain/gcc_solink_wrapper.py" --readelf="readelf" --nm="nm" --sofile="./libpublic.so" --tocfile="./libpublic.so.TOC" --output="./libpublic.so" -- ../../third_party/llvm-build/Release+Asserts/bin/clang++ -shared -fPIC -Wl,-z,noexecstack -Wl,-z,now -Wl,-z,relro -Wl,-z,defs -Wl,--no-as-needed -fuse-ld=lld -Wl,--icf=all -Wl,--color-diagnostics -m64 -rdynamic -nostdlib++ -Wl,-rpath=\$ORIGIN/. -Wl,-rpath-link=. -o "./libpublic.so" -Wl,-soname="libpublic.so" @"./libpublic.so.rsp"
ld.lld: error: duplicate symbol: download::QuarantineFile(base::FilePath const&, GURL const&, GURL const&, std::__Cr::basic_string<char, std::__Cr::char_traits<char>, std::__Cr::allocator<char> > const&)
>>> defined at quarantine.cc:14 (../../components/download/quarantine/quarantine.cc:14)
>>>            quarantine/quarantine.o:(download::QuarantineFile(base::FilePath const&, GURL const&, GURL const&, std::__Cr::basic_string<char, std::__Cr::char_traits<char>, std::__Cr::allocator<char> > const&)) in archive obj/components/download/quarantine/libquarantine.a
>>> defined at quarantine_linux.cc:45 (../../components/download/quarantine/quarantine_linux.cc:45)
>>>            quarantine/quarantine_linux.o:(.text+0x0) in archive obj/components/download/quarantine/libquarantine.a
clang: error: linker command failed with exit code 1 (use -v to see invocation)

It is interesting, that without the debugging these errors had not been happened...

wchen342 commented 3 years ago

The patches in this repo is not adapted for debug builds so this can happen. Just comment out the extra one, it shall be fine.

TCH68k commented 3 years ago

Almost done, but a few undefined symbol popped up at the eleventh hour again...

[13059/13293] SOLINK ./libcontent.so
FAILED: libcontent.so libcontent.so.TOC
python "../../build/toolchain/gcc_solink_wrapper.py" --readelf="readelf" --nm="nm" --sofile="./libcontent.so" --tocfile="./libcontent.so.TOC" --output="./libcontent.so" -- ../../third_party/llvm-build/Release+Asserts/bin/clang++ -shared -fPIC -Wl,-z,noexecstack -Wl,-z,now -Wl,-z,relro -Wl,-z,defs -Wl,--no-as-needed -fuse-ld=lld -Wl,--icf=all -Wl,--color-diagnostics -m64 -rdynamic -nostdlib++ -Wl,-rpath=\$ORIGIN/. -Wl,-rpath-link=. -o "./libcontent.so" -Wl,-soname="libcontent.so" @"./libcontent.so.rsp"
ld.lld: error: undefined symbol: switches::kFingerprintingClientRectsNoise
>>> referenced by runtime_features.cc:0 (../../content/child/runtime_features.cc:0)
>>>               obj/content/child/child/runtime_features.o:(content::SetRuntimeFeaturesDefaultsAndUpdateFromArgs(base::CommandLine const&))

ld.lld: error: undefined symbol: switches::kFingerprintingCanvasMeasureTextNoise
>>> referenced by runtime_features.cc:481 (../../content/child/runtime_features.cc:481)
>>>               obj/content/child/child/runtime_features.o:(content::SetRuntimeFeaturesDefaultsAndUpdateFromArgs(base::CommandLine const&))

ld.lld: error: undefined symbol: switches::kFingerprintingCanvasImageDataNoise
>>> referenced by runtime_features.cc:483 (../../content/child/runtime_features.cc:483)
>>>               obj/content/child/child/runtime_features.o:(content::SetRuntimeFeaturesDefaultsAndUpdateFromArgs(base::CommandLine const&))

ld.lld: error: undefined symbol: switches::kFingerprintingClientRectsNoise
>>> referenced by render_process_host_impl.cc
>>>               obj/content/browser/browser/render_process_host_impl.o:(content::RenderProcessHostImpl::PropagateBrowserCommandLineToRenderer(base::CommandLine const&, base::CommandLine*)::kSwitchNames)

ld.lld: error: undefined symbol: switches::kFingerprintingCanvasMeasureTextNoise
>>> referenced by render_process_host_impl.cc
>>>               obj/content/browser/browser/render_process_host_impl.o:(content::RenderProcessHostImpl::PropagateBrowserCommandLineToRenderer(base::CommandLine const&, base::CommandLine*)::kSwitchNames)

ld.lld: error: undefined symbol: switches::kFingerprintingCanvasImageDataNoise
>>> referenced by render_process_host_impl.cc
>>>               obj/content/browser/browser/render_process_host_impl.o:(content::RenderProcessHostImpl::PropagateBrowserCommandLineToRenderer(base::CommandLine const&, base::CommandLine*)::kSwitchNames)
clang: error: linker command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed.

What is this "fingerprinting" stuff? Is it necessary?

wchen342 commented 3 years ago

This is probably caused by is_debug=true. If the function calls linker is referring to is surrended by IF IS_DEBUG macros then you can just delete them.

It comes from patches/extra/bromite/flag-fingerprinting-canvas-image-data-noise.patch. It adds fingerprinting resistence flags but they are disabled by defualt, so if you don't need those you can just ignore the fingerprinting related patches.

TCH68k commented 3 years ago

Unfortunately it is not the case. No IF IS_DEBUG macros around the calls and the patch you've referred to actually does not adds kFingerprintingClientRectsNoise to anywhere, just adds further calls beside kFingerprintingClientRectsNoise. I've checked where is this function defined and it is third_party/ungoogled/ungoogled_switches.cc:

#include "third_party/ungoogled/ungoogled_switches.h"

namespace switches {

// Enable fingerprinting deception for getClientRects and getBoundingClientRect
const char kFingerprintingClientRectsNoise[] = "fingerprinting-client-rects-noise";

// Enable fingerprinting deception for measureText
const char kFingerprintingCanvasMeasureTextNoise[] = "fingerprinting-canvas-measuretext-noise";

// Enable fingerprinting deception for Canvas image data
const char kFingerprintingCanvasImageDataNoise[] = "fingerprinting-canvas-image-data-noise";

}  // namespace switches

Perhaps not the function but the including of this file's header is surrounded by IF IS_DEBUG macros? Where is this libcontent?

wchen342 commented 3 years ago

The actual file that called the function is in runtime_features.cc as the error messages showed. I am not sure why the linker cannot find the function. Maybe do a search in you BUILD.gn files and see whether ungoogled_switches.h and ungoogled_switches.cc are correctly included? Also check whether these files actually exists and are in correct locations.

TCH68k commented 3 years ago

It is included in runtime_features.cc, but no debug macros there. However, it is not included in content/child/BUILD.gn. I've added these two files into the sources array, but the result is:

[1/1] Regenerating ninja files
ninja: error: '../../content/child/ungoogled_switches.cc', needed by 'obj/content/child/child/ungoogled_switches.o', missing and no known rule to make it

The deps array contains an entry of "//third_party/ungoogled:switches",.

The two files do exists and they are in third_party/ungoogled/. But i don't know how to tell the BUILD.gn to include them properly.

wchen342 commented 3 years ago

That doesn't look right, how did you add the files? Also isn't the dependecy //components/ungoogled:ungoogled_switches?

Can you run

grep -rIEC 5 --exclude-dir=out 'ungoogled.?switches' .

in your src folder and post the results?

TCH68k commented 3 years ago

I added the files by hand into the BUILD.gn's source variable. I have no idea what is the dependency, it was in the deps what i wrote in my last post.

I've ran the command and here is the result: http://oscomp.hu/depot/switches.log

wchen342 commented 3 years ago

Never mind the dependency name. It was the old name and new name changed recently.

Your gn files looks fine to me. I think probably ninja messed things up. Maybe you shall backup the out folder, and then try removing all .ninja files and regenerate using gn?

TCH68k commented 3 years ago

I did so. Backed up the out/Release dir, removed the .ninja files and then ran

gn gen out/Release "--args=`cat flags.gn | tr '\n' ' '`"

again. The result is the same as it was before.

wchen342 commented 3 years ago

That doesn't make sense. ninja shouldn't be looking for content/child/ungoogled_switches.cc unless there is an entry of ungoogled_switches.cc in one of the source array of content/child/BUILD.gn, but I didn't see that from your grep output.

Maybe try just copying the files manually.

TCH68k commented 3 years ago

My bad, i was misunderstandable. So, i've ran

gn gen out/Release "--args=`cat flags.gn | tr '\n' ' '`"

That worked, but then i ran

autoninja -C out/Release chrome

And that resulted in the same error as before:

[1/235] SOLINK ./libcontent.so
FAILED: libcontent.so libcontent.so.TOC
python "../../build/toolchain/gcc_solink_wrapper.py" --readelf="readelf" --nm="nm" --sofile="./libcontent.so" --tocfile="./libcontent.so.TOC" --output="./libcontent.so" -- ../../third_party/llvm-build/Release+Asserts/bin/clang++ -shared -fPIC -Wl,-z,noexecstack -Wl,-z,now -Wl,-z,relro -Wl,-z,defs -Wl,--no-as-needed -fuse-ld=lld -Wl,--icf=all -Wl,--color-diagnostics -m64 -rdynamic -nostdlib++ -Wl,-rpath=\$ORIGIN/. -Wl,-rpath-link=. -o "./libcontent.so" -Wl,-soname="libcontent.so" @"./libcontent.so.rsp"
ld.lld: error: undefined symbol: switches::kFingerprintingClientRectsNoise
>>> referenced by runtime_features.cc:0 (../../content/child/runtime_features.cc:0)
>>>               obj/content/child/child/runtime_features.o:(content::SetRuntimeFeaturesDefaultsAndUpdateFromArgs(base::CommandLine const&))

ld.lld: error: undefined symbol: switches::kFingerprintingCanvasMeasureTextNoise
>>> referenced by runtime_features.cc:481 (../../content/child/runtime_features.cc:481)
>>>               obj/content/child/child/runtime_features.o:(content::SetRuntimeFeaturesDefaultsAndUpdateFromArgs(base::CommandLine const&))

ld.lld: error: undefined symbol: switches::kFingerprintingCanvasImageDataNoise
>>> referenced by runtime_features.cc:483 (../../content/child/runtime_features.cc:483)
>>>               obj/content/child/child/runtime_features.o:(content::SetRuntimeFeaturesDefaultsAndUpdateFromArgs(base::CommandLine const&))

ld.lld: error: undefined symbol: switches::kFingerprintingClientRectsNoise
>>> referenced by render_process_host_impl.cc
>>>               obj/content/browser/browser/render_process_host_impl.o:(content::RenderProcessHostImpl::PropagateBrowserCommandLineToRenderer(base::CommandLine const&, base::CommandLine*)::kSwitchNames)

ld.lld: error: undefined symbol: switches::kFingerprintingCanvasMeasureTextNoise
>>> referenced by render_process_host_impl.cc
>>>               obj/content/browser/browser/render_process_host_impl.o:(content::RenderProcessHostImpl::PropagateBrowserCommandLineToRenderer(base::CommandLine const&, base::CommandLine*)::kSwitchNames)

ld.lld: error: undefined symbol: switches::kFingerprintingCanvasImageDataNoise
>>> referenced by render_process_host_impl.cc
>>>               obj/content/browser/browser/render_process_host_impl.o:(content::RenderProcessHostImpl::PropagateBrowserCommandLineToRenderer(base::CommandLine const&, base::CommandLine*)::kSwitchNames)
clang: error: linker command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed.

As for content/child/BUILD.gn, the variable array deps contains

    "//third_party/ungoogled:switches",

The one, you've said is the old one.

wchen342 commented 3 years ago

I'm running out of ideas..check whether third_party/ungoogled/ungoogled_switches.h is correctly included in content/child/runtime_features.cc and content/browser/renderer_host/render_process_host_impl.cc?

Also once I had a problem with a lld bug, so try a slightly higher or lower version of lld.

TCH68k commented 3 years ago

In both files this line exists:

#include "third_party/ungoogled/ungoogled_switches.h"

As for lld, isn't Chromium's building system supplies it's own of it at third_party/llvm-build/Release+Asserts/bin/lld? (It's 9.0, i've checked it.) I actually have no any own symlink of lld; there is CLang 12's at /usr/lib/llvm-12/bin/lld, but it's not in the $PATH and has not been symlinked as lld at anywhere in my filesystem.

Still, i tried what will happen. I've created the symlinks of lld and ld.lld and renamed the building systems's one to lld1. Result is slightly differently formatted

[1/235] SOLINK ./libcontent.so
FAILED: libcontent.so libcontent.so.TOC
python "../../build/toolchain/gcc_solink_wrapper.py" --readelf="readelf" --nm="nm" --sofile="./libcontent.so" --tocfile="./libcontent.so.TOC" --output="./libcontent.so" -- ../../third_party/llvm-build/Release+Asserts/bin/clang++ -shared -fPIC -Wl,-z,noexecstack -Wl,-z,now -Wl,-z,relro -Wl,-z,defs -Wl,--no-as-needed -fuse-ld=lld -Wl,--icf=all -Wl,--color-diagnostics -m64 -rdynamic -nostdlib++ -Wl,-rpath=\$ORIGIN/. -Wl,-rpath-link=. -o "./libcontent.so" -Wl,-soname="libcontent.so" @"./libcontent.so.rsp"
ld.lld: error: undefined symbol: switches::kFingerprintingClientRectsNoise
>>> referenced by runtime_features.cc:0 (../../content/child/runtime_features.cc:0)
>>>               obj/content/child/child/runtime_features.o:(content::SetRuntimeFeaturesDefaultsAndUpdateFromArgs(base::CommandLine const&))
>>> referenced by render_process_host_impl.cc
>>>               obj/content/browser/browser/render_process_host_impl.o:(content::RenderProcessHostImpl::PropagateBrowserCommandLineToRenderer(base::CommandLine const&, base::CommandLine*)::kSwitchNames)

ld.lld: error: undefined symbol: switches::kFingerprintingCanvasMeasureTextNoise
>>> referenced by runtime_features.cc:481 (../../content/child/runtime_features.cc:481)
>>>               obj/content/child/child/runtime_features.o:(content::SetRuntimeFeaturesDefaultsAndUpdateFromArgs(base::CommandLine const&))
>>> referenced by render_process_host_impl.cc
>>>               obj/content/browser/browser/render_process_host_impl.o:(content::RenderProcessHostImpl::PropagateBrowserCommandLineToRenderer(base::CommandLine const&, base::CommandLine*)::kSwitchNames)

ld.lld: error: undefined symbol: switches::kFingerprintingCanvasImageDataNoise
>>> referenced by runtime_features.cc:483 (../../content/child/runtime_features.cc:483)
>>>               obj/content/child/child/runtime_features.o:(content::SetRuntimeFeaturesDefaultsAndUpdateFromArgs(base::CommandLine const&))
>>> referenced by render_process_host_impl.cc
>>>               obj/content/browser/browser/render_process_host_impl.o:(content::RenderProcessHostImpl::PropagateBrowserCommandLineToRenderer(base::CommandLine const&, base::CommandLine*)::kSwitchNames)
clang: error: linker command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed.

but otherwise identical: three missing symbols (switches::kFingerprintingClientRectsNoise, switches::kFingerprintingCanvasMeasureTextNoise, switches::kFingerprintingCanvasImageDataNoise) in two files (runtime_features.cc, render_process_host_impl.cc).

wchen342 commented 3 years ago

Does ungoogled_switches.o exists at all in out/Default/obj?

TCH68k commented 3 years ago

It does at out/Release/obj/third_party/ungoogled/switches/ungoogled_switches.o and it contains the symbols too:

ungoogled_switches.o:     file format elf64-x86-64

SYMBOL TABLE:
0000000000000000 l    df *ABS*  0000000000000000 ungoogled_switches.cc
0000000000000000 g     O .rodata        0000000000000022 .hidden _ZN8switches31kFingerprintingClientRectsNoiseE
0000000000000060 g     O .rodata        0000000000000027 .hidden _ZN8switches35kFingerprintingCanvasImageDataNoiseE
0000000000000030 g     O .rodata        0000000000000028 .hidden _ZN8switches37kFingerprintingCanvasMeasureTextNoiseE
wchen342 commented 3 years ago

So basically it seems lld doesn't see the file at all. I am not sure whether you can manually add the search path. I have no more ideas now since I am not an expert in cpp.

EDIT: if you run the gcc_solink_wrapper.py line manually with the addition of -rpath to the folder containing ungoogled_switches.o, will it work?

As a last resort, you can probably just ignore the patches about the fingerprinting flags. They are not enabled by default and known to cause problems on some sites, so you can just ignore them.

TCH68k commented 3 years ago

I've ran

python "../../build/toolchain/gcc_solink_wrapper.py" --readelf="readelf" --nm="nm" --sofile="./libcontent.so" --tocfile="./libcontent.so.TOC" --output="./libcontent.so" -- ../../third_party/llvm-build/Release+Asserts/bin/clang++ -shared -fPIC -Wl,-z,noexecstack -Wl,-z,now -Wl,-z,relro -Wl,-z,defs -Wl,--no-as-needed -fuse-ld=lld -Wl,--icf=all -Wl,--color-diagnostics -m64 -rdynamic -nostdlib++ -Wl,-rpath=obj/third_party/ungoogled/switches/ -Wl,-rpath-link=. -o "./libcontent.so" -Wl,-soname="libcontent.so" @"./libcontent.so.rsp"

in out/Release. The result is the same.

I would like to avoid restarting the entire building... Can i revoke the patches somehow and force the buildsystem to build only what has changed? Also, what exactly these fingerprinting patches about?

wchen342 commented 3 years ago

Actually it should be rpath-link, sorry.

If you know the basic knowledge about C++ then you can just remove the #include and the mention to those variables causing the problem.

The patches added flags you can enable to add random noise to Canvas, which can potentially be used by website to uniquely identify your browser. But there are extensions does this too, like Privacy Badger.

TCH68k commented 3 years ago

Tried that too already, same result.

I did so, now only the final binary cannot be linked because crc32 is a missing symbol.

[240/240] LINK ./chrome
FAILED: chrome
python "../../build/toolchain/gcc_link_wrapper.py" --output="./chrome" -- ../../third_party/llvm-build/Release+Asserts/bin/clang++ -fPIC -Wl,-z,noexecstack -Wl,-z,now -Wl,-z,relro -Wl,-z,defs -Wl,--no-as-needed -fuse-ld=lld -Wl,--icf=all -Wl,--color-diagnostics -m64 -rdynamic -nostdlib++ -pie -Wl,-rpath-link=. -Wl,--disable-new-dtags -Wl,-rpath=\$ORIGIN/. -Wl,-rpath-link=. -o "./chrome" -Wl,--start-group @"./chrome.rsp" ./libembedder.so ./libbase.so ./libbindings.so ./libbindings_base.so ./libmojo_public_system_cpp.so ./libmojo_public_system.so ./libmojo_cpp_platform.so ./libmessage_support.so ./libmojo_mojom_bindings.so ./libmojo_mojom_bindings_shared.so ./liburl.so ./libmojo_base_mojom_shared.so ./libmojo_base_mojom.so ./libmojo_base_lib.so ./libmojo_base_shared_typemap_traits.so ./libbase_i18n.so ./libicui18n.so ./libicuuc.so ./libnet.so ./libprotobuf_lite.so ./libcrcrypto.so ./libboringssl.so ./libipc.so ./libipc_mojom.so ./libipc_mojom_shared.so ./libcontent_public_common_mojo_bindings_shared.so ./libblink_mojo_bindings_shared.so ./libblink_android_mojo_bindings_shared.so ./libweb_feature_mojo_bindings_mojom_shared.so ./libservice_manager_mojom_shared.so ./libservice_manager_mojom_constants_shared.so ./libmojom_platform_shared.so ./libandroid_mojo_bindings_shared.so ./libspeech_recognition_error_code_mojom_shared.so ./libusb_shared.so ./libui_accessibility_ax_mojom_shared.so ./libgeometry.so ./libcontent.so ./libgpu.so ./libgfx.so ./libcolor_space.so ./libskia.so ./libgeometry_skia.so ./libgfx_switches.so ./libanimation.so ./libcodec.so ./librange.so ./libcc_paint.so ./libcc_base.so ./libcc_debug.so ./libcrash_key.so ./libgfx_ipc.so ./libgfx_ipc_geometry.so ./liburl_ipc.so ./libviz_common.so ./libviz_resource_format_utils.so ./libviz_vulkan_context_provider.so ./libui_base.so ./libui_data_pack.so ./libevents_base.so ./libplatform.so ./libkeycodes_x11.so ./libui_base_x.so ./libdisplay.so ./libdisplay_types.so ./libmedia.so ./libshared_memory_support.so ./libkeyed_service_core.so ./libleveldatabase.so ./libservice_manager_mojom.so ./libservice_manager_mojom_constants.so ./libservice_manager_cpp_types.so ./libservice_manager_mojom_traits.so ./libservice_manager_cpp.so ./libmetrics_cpp.so ./libcc.so ./libevents.so ./libui_base_ime.so ./libui_base_ime_types.so ./libui_base_ime_linux.so ./libblink_common.so ./libnetwork_cpp.so ./libnetwork_cpp_base.so ./libmojom_modules_shared.so ./libmojom_core_shared.so ./libspeech_recognition_error_code_mojom.so ./libweb_feature_mojo_bindings_mojom.so ./libui_accessibility_ax_mojom.so ./libcontent_common_mojo_bindings_shared.so ./libmedia_session_cpp.so ./libgfx_ipc_color.so ./libgfx_ipc_skia.so ./libcapture_base.so ./libaccessibility.so ./libevents_x.so ./libgfx_x11.so ./libaura.so ./libcompositor.so ./libsurface.so ./libnetwork_service.so ./libsandbox.so ./libembedder_switches.so ./libmemory_instrumentation.so ./libresource_coordinator_public_mojom.so ./libresource_coordinator_public_mojom_shared.so ./libresource_coordinator_cpp_base.so ./libstorage_common.so ./libpublic.so ./libinterfaces_shared.so ./libresource_coordinator_cpp.so ./libresource_coordinator_cpp_features.so ./libstorage_browser.so ./libchrome_features.so ./libprefs.so ./libos_crypt.so ./libwebdata_common.so ./libsql.so ./libcloud_policy_proto_generated_compile.so ./libpolicy_component.so ./libpolicy_proto.so ./libcommon.so ./libnetwork_session_configurator.so ./libchromium_sqlite3.so ./libnative_theme.so ./libgl_wrapper.so ./libui_message_center_cpp.so ./libppapi_shared.so ./liburl_matcher.so ./libbluetooth.so ./libdevice_event_log.so ./libdevice_base.so ./libprinting.so ./libsandbox_services.so ./libkeyed_service_content.so ./libsessions.so ./libshell_dialogs.so ./libdbus.so ./libonc.so ./libhost.so ./libviews.so ./libui_base_clipboard.so ./libui_base_clipboard_types.so ./libuser_prefs.so ./libgcm.so ./libui_touch_selection.so ./libproxy_config.so ./libfido.so ./libmanager.so ./libmessage_center.so ./libfontconfig.so ./libx11_events_platform.so ./libdevices.so ./libevents_devices_x11.so ./libffmpeg.so ./libwebview.so ./libdomain_reliability.so ./libstartup_tracing.so ./libui_devtools.so ./libuser_manager.so ./libgesture_detection.so ./libsnapshot.so ./libweb_dialogs.so ./libsuid_sandbox_client.so ./libseccomp_bpf.so ./libdevice_features.so ./libgtk2ui.so ./libbrowser_ui_views.so ./libwm.so ./libui_views_mus_lib.so ./libgles2.so ./libgles2_implementation.so ./libraster.so ./libwebgpu.so ./libwm_public.so ./libppapi_proxy.so ./libcaptive_portal.so ./libcertificate_matching.so ./libextras.so ./liblive_tab_count_metrics.so ./libcapture_lib.so ./libmirroring_service.so ./libmidi.so ./libmedia_mojo_services.so ./libmedia_gpu.so ./libmedia_webrtc.so ./libmojo_core_embedder.so ./libswitches.so ./libzygote.so ./libsimple_browser.so ./libvr_common.so ./libvr_base.so ./libtracing.so ./libppapi_host.so ./libblink_controller.so ./libblink_core.so ./libblink_core_mojo_bindings_shared.so ./libwtf.so ./libmojo_base_mojom_blink.so ./libservice_manager_mojom_blink.so ./libservice_manager_mojom_constants_blink.so ./libweb_feature_mojo_bindings_mojom_blink.so ./libblink_platform.so ./libcc_animation.so ./libresource_coordinator_public_mojom_blink.so ./libspeech_recognition_error_code_mojom_blink.so ./libv8.so ./libgin.so ./libblink_modules.so ./libnet_with_v8.so ./libcatalog_lib.so ./libdiscardable_memory_client.so ./libfreetype_harfbuzz.so ./libv8_libplatform.so ./libheadless.so ./libc++.so -Wl,--end-group   -ldl -lpthread -lrt -lX11 -lX11-xcb -lxcb -lXcomposite -lXcursor -lXdamage -lXext -lXfixes -lXi -lXrender -lXtst -lgmodule-2.0 -lgobject-2.0 -lgthread-2.0 -lglib-2.0 -lnss3 -lnssutil3 -lsmime3 -lplds4 -lplc4 -lnspr4 -lcups -ldbus-1 -lexpat -lgio-2.0 -lopenjp2
ld.lld: error: undefined symbol: crc32
>>> referenced by crc.cc:28 (../../courgette/crc.cc:28)
>>>               obj/courgette/courgette_common/crc.o:(courgette::CalculateCrc(unsigned char const*, unsigned long))
clang: error: linker command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed.

Thanks, so i'd rather do not use the patches, but install Privacy Badger or similar. Or maybe it is possible to globally disable the Canvas for good?

wchen342 commented 3 years ago

Does this and this help?

I don't remember there is a flag to disable canvas. Canvas is used on a large amount of websites so even if you can disable it most of the websites won't work without it.

TCH68k commented 3 years ago

Yes, the second one helped:

sed -i.bak 's/        ldflags = \[\]/        ldflags = \[ "-lz" \]/g' chrome/BUILD.gn

The chrome binary has been successfully built. And it works, it does not crashes at all. So this means, that one of the removed patches caused the crashes. Or maybe the debug mode prevented it? Tomorrow i'll build Chromium again with no debug and without the libevent, libicu and fingerprinting patches and see if it works.

Then Privacy Badger will do the job.

TCH68k commented 3 years ago

I've just finished building the chrome binary with the flags.gn, without the patches above and without debug. It still crashes:

Fontconfig warning: "/etc/fonts/fonts.conf", line 100: unknown element "blank"
[26697:26697:0117/171906.350669:ERROR:browser_dm_token_storage_linux.cc(101)] Error: /etc/machine-id contains 0 characters (32 were expected).
[26697:26827:0117/171906.476505:ERROR:object_proxy.cc(621)] Failed to call method: org.freedesktop.Notifications.GetCapabilities: object_path= /org/freedesktop/Notifications: org.freedesktop.DBus.Error.ServiceUnknown: The name org.freedesktop.Notifications was not provided by any .service files
Érvénytelen utasítás

(The last line means "invalid instruction" in hungarian.) These messages (with the exception of the invalid instruction) are printed out by my working version too, so they are not the cause.

So, the debug build does not crash, the identically parametered release build does. We cannot debug. Can i do anything else, but remove the lines from the flags.gn one by one and build it again and again to test which causes the problem?

wchen342 commented 3 years ago

Did you include is_official_build=true? Try remove that.

is_debug only disables certain compiler optimizations and enables some extra checks in the code. The functionalities shall be the same,so if speed is not a concern then you can just use a debug build.

EDIT: Also, in flags.gn change to exclude_unwind_tables=false, symbol_level=1 and blink_symbol_level=1 can probably makes it print crash backtrace even in non-debug build.

TCH68k commented 3 years ago

And bingo! With is_official_build set to false it now works flawlessly, no further debugging needed.

So, thank you very much again for your vast amount of help and tremendous patience, for in round two too. Here is the - now really - final script to build (it is just finished running and the resulting package contains everything, the binary is working):

#!/bin/sh
cd "$HOME"
mkdir kocsog_gugli 2>/dev/null
cd kocsog_gugli

echo
echo "======================"
echo "Getting depot tools..."
echo "======================"
echo
git clone https://chromium.googlesource.com/chromium/tools/depot_tools.git
export PATH=$PATH:"$HOME"/kocsog_gugli/depot_tools

echo
echo "=========================="
echo "Getting Chromium source..."
echo "=========================="
echo
mkdir chromium 2>/dev/null
cd chromium/
fetch --nohooks chromium
free > /dev/null && sync > /dev/null && echo 3 > /proc/sys/vm/drop_caches && free > /dev/null

echo
echo "===================="
echo "Checkout and sync..."
echo "===================="
echo
cd src
git checkout 73.0.3683.75
gclient sync -D
free > /dev/null && sync > /dev/null && echo 3 > /proc/sys/vm/drop_caches && free > /dev/null

# Use the "fontconfig.patch" from UGC and not from Debian. The rest of the two patchset can be merged.
echo
echo "==========="
echo "Patching..."
echo "==========="
echo
echo "Getting Debian patchset..."
wget http://deb.debian.org/debian/pool/main/c/chromium/chromium_73.0.3683.75-1~deb9u1.debian.tar.xz
tar xf chromium_73.0.3683.75-1~deb9u1.debian.tar.xz
rm chromium_73.0.3683.75-1~deb9u1.debian.tar.xz
rm -rf debian/patches/arm
find debian/patches/ -name '*.patch' -exec mv {} ./ \;
rm -rf debian
echo "Getting Ungoogled-Chromium patchset..."
git clone https://github.com/Eloston/ungoogled-chromium/
cd ungoogled-chromium/
git checkout 73.0.3683.103-2
cd ..
find ungoogled-chromium/patches/ -name '*.patch' -exec mv {} ./ \;
rm -rf ungoogled-chromium
# Remove unneeded libevent, libicu and fingerprinting patches.
rm event.patch icu.patch icu57.patch icu-new.patch fingerprinting-flags-client-rects-and-measuretext.patch flag-fingerprinting-canvas-image-data-noise.patch
echo "Applying patches..."
for PATCH in *.patch
do
    echo "****************"
    echo "* ""$PATCH"
    echo "****************"
    patch -p1 -f < "$PATCH"
    echo "****************"
    echo
done
rm *.patch
# Some files are left in an uncompilable state.
echo "Some hand patching..."
# Designated initializers patch is erroneous with the bundled CLang 9.
sed -i.bak 's/    Settings(bool randomise):randomised_sampling_enabled(randomise) {}//g' base/task/sequence_manager/sequence_manager.h
# Patch a mispatched file's macro.
sed -i.bak -n '1h;1!H;${g;s/#endif\n#endif/#endif/;p;}' chrome/browser/ui/chrome_pages.cc
# Safe browsing patch is erroneous with chrome 73.0.
sed -i.bak '/^static_library("safe_browsing") {/i sources = []\n deps = []' chrome/browser/safe_browsing/BUILD.gn
# Comment some ICU calls in the bug reporting.
sed -i.bak 's|u_|// u_|g' third_party/breakpad/breakpad/src/common/string_conversion.cc
# Rename a duplicate dummy function.
sed -i.bak 's/QuarantineFileResult QuarantineFile/QuarantineFileResult QuarantineFile1/g' components/download/quarantine/quarantine.cc
# Chromium needs libz.
sed -i.bak 's/        ldflags = \[\]/        ldflags = \[ "-lz" \]/g' chrome/BUILD.gn

echo
echo "============="
echo "Generating..."
echo "============="
echo
export PKG_CONFIG_PATH="$PKG_CONFIG_PATH":/usr/lib/x86_64-linux-gnu/pkgconfig/
wget https://raw.githubusercontent.com/Eloston/ungoogled-chromium/73.0.3683.103-2/flags.gn
echo 'gtk_version=2' >> flags.gn
echo 'use_gnome_keyring=false' >> flags.gn
echo 'use_pulseaudio=false' >> flags.gn
echo 'use_sysroot=false' >> flags.gn
sed -i.bak 's/is_official_build=true/is_official_build=false/g' flags.gn
gn gen out/Release "--args=`cat flags.gn | tr '\n' ' '`"
gn args --list out/Release/ > ninja.settings
free > /dev/null && sync > /dev/null && echo 3 > /proc/sys/vm/drop_caches && free > /dev/null

echo
echo "==========="
echo "Building..."
echo "==========="
echo
autoninja -C out/Release chrome
free > /dev/null && sync > /dev/null && echo 3 > /proc/sys/vm/drop_caches && free > /dev/null

echo
echo "============"
echo "Packaging..."
echo "============"
echo
cd out/Release
mkdir ungoogled-chromium-devuan2-gtk2-portable
cp -R locales chrome *.pak *.info *.so *.bin *.dat ungoogled-chromium-devuan2-gtk2-portable/
tar -zcf ungoogled-chromium-devuan2-gtk2-portable.tar.gz ungoogled-chromium-devuan2-gtk2-portable
rm -rf ungoogled-chromium-devuan2-gtk2-portable
mv ungoogled-chromium-devuan2-gtk2-portable.tar.gz "$HOME"/
free > /dev/null && sync > /dev/null && echo 3 > /proc/sys/vm/drop_caches && free > /dev/null

if [ "$1" != "" ];
then
    echo
    echo "=============="
    echo "Cleaning up..."
    echo "=============="
    echo
    rm -rf "$HOME"/kocsog_gugli
    free > /dev/null && sync > /dev/null && echo 3 > /proc/sys/vm/drop_caches && free > /dev/null
fi
wchen342 commented 3 years ago

Glad to hear it finally worked for you!

If you have some extra time, probably you can summarize the pitfalls you experienced and how you solved them in the wiki. It can be helpful to others. But no worry if you don't have the time, since this thread itself can be a great reference for those who facing building problems in the future.

TCH68k commented 3 years ago

Okay, when i'll get some free time for this, i will summarize them. However, should i create a new article or add the content to another?

wchen342 commented 3 years ago

Considering the length of the thread it shall be better to be a separate article. You can open a PR to the wiki repo and see what Eloston says. I think most likely he will agree a new article is good.

TCH68k commented 3 years ago

Very well, i'll collect my thoughts, re-read the entire commentstream and make a new article about building UGC with GTK2. For now, under Devuan 2 (Debian 9), but when i'll have to update to Devuan 3 (Debian 10), i'll cover that too. Whenever i get the time. (Since Debian 10 is officially supported by UGC, it will be much easier, i just have to use the GTK2 patch...)

I'm curious if the GTK2 patch could be adapted to later Chromium versions too...

PF4Public commented 2 years ago

Closing this issue in favour of #1589 Please continue this discussion in #1589 if you have something to add.