Closed Ahrotahn closed 2 weeks ago
I had some problems with a missing file (components/optimization_guide/machine_learning_tflite_buildflags.h
) and a BUILDFLAG
define (BUILDFLAG(BUILD_WITH_TFLITE_LIB)
; seems to be a windows specific problem) which resulted in the following patch (which includes the core repo patch as well):
--- a/services/passage_embeddings/passage_embeddings_service.cc
+++ b/services/passage_embeddings/passage_embeddings_service.cc
@@ -5,8 +5,6 @@
#include "services/passage_embeddings/passage_embeddings_service.h"
#include "base/files/file.h"
-#include "components/optimization_guide/machine_learning_tflite_buildflags.h"
-#include "services/passage_embeddings/passage_embedder.h"
namespace passage_embeddings {
@@ -21,7 +19,7 @@ void PassageEmbeddingsService::LoadModels(
mojom::PassageEmbeddingsLoadModelsParamsPtr params,
mojo::PendingReceiver<mojom::PassageEmbedder> model,
LoadModelsCallback callback) {
-#if BUILDFLAG(BUILD_WITH_TFLITE_LIB)
+#if (0)
embedder_ = std::make_unique<PassageEmbedder>(std::move(model));
// Load the model files.
--- a/services/passage_embeddings/passage_embeddings_service.h
+++ b/services/passage_embeddings/passage_embeddings_service.h
@@ -5,13 +5,11 @@
#ifndef SERVICES_PASSAGE_EMBEDDINGS_PASSAGE_EMBEDDINGS_SERVICE_H_
#define SERVICES_PASSAGE_EMBEDDINGS_PASSAGE_EMBEDDINGS_SERVICE_H_
-#include "components/optimization_guide/machine_learning_tflite_buildflags.h"
#include "mojo/public/cpp/bindings/receiver.h"
#include "services/passage_embeddings/public/mojom/passage_embeddings.mojom.h"
namespace passage_embeddings {
-class PassageEmbedder;
// Class implementation of the passage embeddings service mojo interface.
class PassageEmbeddingsService : public mojom::PassageEmbeddingsService {
@@ -30,7 +28,7 @@ class PassageEmbeddingsService : public mojom::PassageEmbeddingsService {
mojo::Receiver<mojom::PassageEmbeddingsService> receiver_;
-#if BUILDFLAG(BUILD_WITH_TFLITE_LIB)
+#if (0)
std::unique_ptr<PassageEmbedder> embedder_;
#endif
};
Now I am stuck with another windows specific problem:
In file included from ../../services/webnn/dml/adapter.cc:5:
In file included from ../..\services/webnn/dml/adapter.h:14:
../..\third_party/microsoft_dxheaders/include/directml.h(9,10): fatal error: 'third_party/microsoft_dxheaders/src/include/directx/DirectML.h' file not found
9 | #include "third_party/microsoft_dxheaders/src/include/directx/DirectML.h"
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The header is included by a few files and I am not sure if I sould patch all of them or if there is an other cause for the problem.
Can you check if you're building with build_with_tflite_lib=false
set? Those defines should already be handled by that.
Yes, build_with_tflite_lib=false
is set. For windows we use the flags from this repo together with a custom list of flags.
build_with_tflite_lib=false
chrome_pgo_phase=0
clang_use_chrome_plugins=false
disable_fieldtrial_testing_config=true
enable_hangout_services_extension=false
enable_mdns=false
enable_nacl=false
enable_reading_list=false
enable_remoting=false
enable_reporting=false
enable_service_discovery=false
enable_widevine=true
exclude_unwind_tables=true
google_api_key=""
google_default_client_id=""
google_default_client_secret=""
safe_browsing_mode=0
treat_warnings_as_errors=false
use_official_google_api_keys=false
use_unofficial_version_number=false
chrome_pgo_phase=0
enable_swiftshader=false
ffmpeg_branding="Chrome"
is_clang=true
is_component_build=false
is_debug=false
is_official_build=true
proprietary_codecs=true
target_cpu="x64"
use_sysroot=false
dcheck_always_on=false
blink_symbol_level=0
v8_symbol_level=0
symbol_level=0
enable_rust=true
enable_mse_mpeg2ts_stream_parser=true
It looks like this could be another upstream out-of-order dependency issue. The BUILD.gn for passage_embeddings_service.cc has the generation of the very header that defines the buildflag gated behind the gn flag... So if you weren't lucky enough to have it generated already by some other non-gated section (like here for example) then the header wouldn't exist when you need it. See if this change helps:
--- a/services/passage_embeddings/BUILD.gn
+++ b/services/passage_embeddings/BUILD.gn
@@ -31,6 +31,7 @@
}
deps = [
+ "//components/optimization_guide:machine_learning_tflite_buildflags",
"//services/passage_embeddings/public/mojom",
"//third_party/sentencepiece:sentencepiece",
"//third_party/sentencepiece:sentencepiece_proto",
@@ -38,7 +39,6 @@
if (build_with_tflite_lib) {
deps += [
- "//components/optimization_guide:machine_learning_tflite_buildflags",
"//components/optimization_guide/core:machine_learning",
]
}
This indeed solves the first problem.
The second problem however still exists, the file third_party/microsoft_dxheaders/src/include/directx/DirectML.h
is not found. This is the content of the third_party/microsoft_dxheaders
folder:
$ find third_party/microsoft_dxheaders
third_party/microsoft_dxheaders/include
third_party/microsoft_dxheaders/include/directml.h
third_party/microsoft_dxheaders/LICENSE
third_party/microsoft_dxheaders/OWNERS
third_party/microsoft_dxheaders/README.chromium
I have no idea how to fix that. I could patch all occurances of the include but I am not sure if that really solves the issue.
This is the content of the
third_party/microsoft_dxheaders
folder:
Could something be missing from the tarball?
They decided to use third-party directx headers in [5514432] so they could use ML features that haven't shipped in windows yet. The tarballs are created on posix systems and will not include those headers, so there's a few ways we can approach this:
We could add a new patch to build without webnn. The changes, currently at least, are fairly minimal:
Also, I'm starting to think that it might be a good idea to create a patch in the core repo for those dependency issues. I haven't seen any movement on their end to update to a newer version of ninja yet, and this is affecting multiple platforms now as well. It's a weird situation because it would be fixing upstream issues, so I'm not sure where we'd place such a patch. Maybe we could create upstream-fixes
alongside the core
and extra
directories? Would it make more sense for that to instead be under core
?
Maybe we could create
upstream-fixes
alongside thecore
andextra
directories? Would it make more sense for that to instead be undercore
?
Absolutely! Or perhaps mere fixes
to spare some bytes :)
The tarballs are created on posix systems and will not include those headers
will never include those headers? Is it their omission that may eventually be fixed by them?
They decided to use third-party directx headers in [5514432] so they could use ML features that haven't shipped in windows yet. The tarballs are created on posix systems and will not include those headers
Sounds like we will not be able to build from tarball on windows unless we completely remove the feature.
Also, I'm starting to think that it might be a good idea to create a patch in the core repo for those dependency issues. I haven't seen any movement on their end to update to a newer version of ninja yet, and this is affecting multiple platforms now as well. It's a weird situation because it would be fixing upstream issues, so I'm not sure where we'd place such a patch. Maybe we could create
upstream-fixes
alongside thecore
andextra
directories? Would it make more sense for that to instead be undercore
?
Sounds like a good idea to me 👍
will never include those headers? Is it their omission that may eventually be fixed by them?
The headers are a windows-only dependency and wouldn't be included in the tarballs unless someone on their end manually intervenes to include them, which I don't believe they would do. Google doesn't use the tarballs and had at one point wanted to deprecate their creation, but many linux distros build with them and convinced Google to keep generating them.
Sounds like we will not be able to build from tarball on windows unless we completely remove the feature.
The more I think about it the more I think building without webnn should be how we proceed. In the best case scenario that api would just allow sites to offload their server hardware costs onto their users. In the worst case scenario malicious sites could leverage users' hardware for their own training, generation, etc.
I'll update the PR in a bit to include both changes.
I think building without webnn should be how we proceed
Should this be mentioned somewhere (in README)?
Maybe we need to create a new document that lists more detail about the differences compared to vanilla Chromium. There are other ramifications due to building without tflite and I know we also recently disabled fedcm. It might be a good place for info about why translations and captions don't work as well.
Builds and runs fine on windows now. I had to update the windows safebrowsing patch because I ran into two linking errors. In case someone runs into one of the following linking errors, here is how I fixed them:
Error:
lld-link: error: undefined symbol: bool __cdecl safe_browsing::IsConsumerScanRequest(class safe_browsing::BinaryUploadService::Request const &)
>>> referenced by ../../chrome/browser/enterprise/connectors/common.cc
>>> obj/chrome/browser\browser/common.obj
Fix:
--- a/chrome/browser/enterprise/connectors/common.cc
+++ b/chrome/browser/enterprise/connectors/common.cc
@@ -571,16 +571,6 @@ void ShowDownloadReviewDialog(const std::u16string& filename,
/* file_count */ 1, state, download_item);
}
-bool IsResumableUpload(const BinaryUploadService::Request& request) {
- // Currently resumable upload doesn't support paste or LBUS. If one day we do,
- // we should update the logic here as well.
- return !safe_browsing::IsConsumerScanRequest(request) &&
- request.cloud_or_local_settings().is_cloud_analysis() &&
- request.content_analysis_request().analysis_connector() !=
- enterprise_connectors::AnalysisConnector::BULK_DATA_ENTRY &&
- IsResumableUploadEnabled();
-}
-
bool CloudMultipartResultIsFailure(BinaryUploadService::Result result) {
return result != BinaryUploadService::Result::SUCCESS;
}
Error:
lld-link: error: undefined symbol: public: __cdecl enterprise_connectors::ContentAnalysisDelegate::Data::~Data(void)
>>> referenced by ../../chrome/browser/printing/print_view_manager_base.cc
>>> obj/chrome/browser\browser/print_view_manager_base.obj
Fix:
--- a/chrome/browser/enterprise/connectors/analysis/content_analysis_delegate.h
+++ b/chrome/browser/enterprise/connectors/analysis/content_analysis_delegate.h
@@ -94,7 +94,7 @@ class ContentAnalysisDelegate : public ContentAnalysisDelegateBase {
Data();
Data(Data&& other);
Data& operator=(Data&& other);
- ~Data();
+ ~Data() = default;
// Helper function to populate `text` and `image` with the data in a
// `content::ClipboardPasteData` object.
While trying to fix the issues I stumbled upon the ENTERPRISE_CONTENT_ANALYSIS
buildflag. It seemed to be everywhere and trying to remove it caused even more errors. I am glad I found a quite simple solution for my linking problems and I think it will haunt us in again the future...
Thanks, I updated the PR with the changes but kept the function in case that gets use later. The enterprise connectors have been a pain for a while, I'm hoping these latest changes will make things easier in the future.
portable linux builds fine and runs at least on manjaro and ubuntu20.04/24.04
Gentoo builds and runs (although I started the build at df2dff971ba848d7bae24b177ef7eceef587731b)
Currently having this issue on macOS:
[8209/60916] LINK ./transport_security_state_generator
FAILED: transport_security_state_generator transport_security_state_generator.dSYM/Contents/Info.plist transport_security_state_generator.dSYM/Contents/Resources/DWARF/transport_security_state_generator
TOOL_VERSION=1718139546 ../../build/toolchain/apple/linker_driver.py -Wcrl,driver,../../third_party/llvm-build/Release+Asserts/bin/clang++ -Wcrl,strippath,/Applications/Xcode-16.0.0-Beta.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/strip -Wcrl,installnametoolpath,../../third_party/llvm-build/Release+Asserts/bin/llvm-install-name-tool -Wcrl,dsym,. -Wcrl,dsymutilpath,/usr/bin/dsymutil -fuse-ld=lld -Wl,--icf=all -Wl,--color-diagnostics -Wl,-mllvm,-instcombine-lower-dbg-declare=0 -Wl,-mllvm,-split-threshold-for-reg-with-hint=0 -Wl,-cache_path_lto,thinlto-cache -Wl,--thinlto-cache-policy=cache_size=10\%:cache_size_bytes=40g:cache_size_files=100000 -flto=thin -Wl,--thinlto-jobs=all -Wl,-mllvm,-import-instr-limit=30 -Wcrl,object_path_lto -Wl,-mllvm,-disable-auto-upgrade-debug-info -fwhole-program-vtables -Wl,--strict-auto-link -Wl,--no-call-graph-profile-sort --target=arm64-apple-macos -Wl,-mllvm,-enable-machine-outliner=never -no-canonical-prefixes -Wl,-dead_strip -Wl,-no_data_in_code_info -Wl,-no_function_starts -nostdlib++ -isysroot ../../../../../../../Applications/Xcode-16.0.0-Beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX15.0.sdk -mmacos-version-min=10.15 -Wl,--lto-O0 -Wl,-ObjC -Wcrl,strip,-x,-S prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/libstd.rlib prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/liballoc.rlib prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/libcfg_if.rlib prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/libcompiler_builtins.rlib prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/libcore.rlib prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/libgetopts.rlib prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/libhashbrown.rlib prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/liblibc.rlib prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/libpanic_abort.rlib prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/libpanic_unwind.rlib prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/librustc_demangle.rlib prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/libstd_detect.rlib prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/libtest.rlib prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/libunicode_width.rlib prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/libunwind.rlib prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/libaddr2line.rlib prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/libadler.rlib prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/libgimli.rlib prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/libmemchr.rlib prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/libminiz_oxide.rlib prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/libobject.rlib -o "./transport_security_state_generator" "@./transport_security_state_generator.rsp" obj/third_party/rust/cxx/v1/lib/libcxx_lib.rlib
ld64.lld: error: undefined symbol: __rust_alloc
>>> referenced by raw_vec.rs:0 (library/alloc/src/raw_vec.rs:0)
>>> prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/liballoc.rlib(alloc-d703698c9307fab6.alloc.94db28bdc29c15e2-cgu.0.rcgu.o):(symbol alloc::raw_vec::finish_grow::hd8e720bf49ed515d+0x58)
>>> referenced by alloc.rs:100 (library/alloc/src/alloc.rs:100)
>>> prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/liballoc.rlib(alloc-d703698c9307fab6.alloc.94db28bdc29c15e2-cgu.0.rcgu.o):(symbol _$LT$$RF$$u5b$u8$u5d$$u20$as$u20$alloc..ffi..c_str..CString..new..SpecNewImpl$GT$::spec_new_impl::he983fa359e3d39e3+0x50)
>>> referenced by alloc.rs:100 (library/alloc/src/alloc.rs:100)
>>> prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/liballoc.rlib(alloc-d703698c9307fab6.alloc.94db28bdc29c15e2-cgu.0.rcgu.o):(symbol alloc::string::String::from_utf8_lossy::h24d159c7d6ec376e+0x68)
>>> referenced 53 more times
ld64.lld: error: undefined symbol: __rust_no_alloc_shim_is_unstable
>>> referenced by raw_vec.rs:0 (library/alloc/src/raw_vec.rs:0)
>>> prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/liballoc.rlib(alloc-d703698c9307fab6.alloc.94db28bdc29c15e2-cgu.0.rcgu.o):(symbol alloc::raw_vec::finish_grow::hd8e720bf49ed515d+0x48)
>>> referenced by raw_vec.rs:0 (library/alloc/src/raw_vec.rs:0)
>>> prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/liballoc.rlib(alloc-d703698c9307fab6.alloc.94db28bdc29c15e2-cgu.0.rcgu.o):(symbol alloc::raw_vec::finish_grow::hd8e720bf49ed515d+0x44)
>>> referenced by mod.rs:1678 (library/core/src/ptr/mod.rs:1678)
>>> prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/liballoc.rlib(alloc-d703698c9307fab6.alloc.94db28bdc29c15e2-cgu.0.rcgu.o):(symbol _$LT$$RF$$u5b$u8$u5d$$u20$as$u20$alloc..ffi..c_str..CString..new..SpecNewImpl$GT$::spec_new_impl::he983fa359e3d39e3+0x3c)
>>> referenced 101 more times
ld64.lld: error: undefined symbol: __rust_realloc
>>> referenced by alloc.rs:138 (library/alloc/src/alloc.rs:138)
>>> prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/liballoc.rlib(alloc-d703698c9307fab6.alloc.94db28bdc29c15e2-cgu.0.rcgu.o):(symbol alloc::raw_vec::finish_grow::hd8e720bf49ed515d+0x34)
>>> referenced by alloc.rs:138 (library/alloc/src/alloc.rs:138)
>>> prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/liballoc.rlib(alloc-d703698c9307fab6.alloc.94db28bdc29c15e2-cgu.0.rcgu.o):(symbol alloc::ffi::c_str::CString::_from_vec_unchecked::hddacc649dd1f027c+0xb8)
>>> referenced by prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/libgimli.rlib(gimli-8cd07c21dd4e4cf2.gimli.ff2f875b2d72cc5f-cgu.0.rcgu.o):(symbol alloc::raw_vec::finish_grow::hcbbdcc52d5575369+0x3c)
>>> referenced 14 more times
ld64.lld: error: undefined symbol: __rust_alloc_error_handler
>>> referenced by alloc.rs:385 (library/alloc/src/alloc.rs:385)
>>> prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/liballoc.rlib(alloc-d703698c9307fab6.alloc.94db28bdc29c15e2-cgu.0.rcgu.o):(symbol alloc::alloc::handle_alloc_error::haab0890707c490e2+0x14)
ld64.lld: error: undefined symbol: __rust_dealloc
>>> referenced by alloc.rs:119 (library/alloc/src/alloc.rs:119)
>>> prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/liballoc.rlib(alloc-d703698c9307fab6.alloc.94db28bdc29c15e2-cgu.0.rcgu.o):(symbol _$LT$$RF$$u5b$u8$u5d$$u20$as$u20$alloc..ffi..c_str..CString..new..SpecNewImpl$GT$::spec_new_impl::he983fa359e3d39e3+0x130)
>>> referenced by c_str.rs:0 (library/alloc/src/ffi/c_str.rs:0)
>>> prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/liballoc.rlib(alloc-d703698c9307fab6.alloc.94db28bdc29c15e2-cgu.0.rcgu.o):(symbol alloc::ffi::c_str::CString::_from_vec_unchecked::hddacc649dd1f027c+0x13c)
>>> referenced by alloc.rs:119 (library/alloc/src/alloc.rs:119)
>>> prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/liballoc.rlib(alloc-d703698c9307fab6.alloc.94db28bdc29c15e2-cgu.0.rcgu.o):(symbol alloc::ffi::c_str::CString::_from_vec_unchecked::hddacc649dd1f027c+0xe4)
>>> referenced 292 more times
clang++: error: linker command failed with exit code 1 (use -v to see invocation)
Traceback (most recent call last):
File "/Users/cubik65536/ungoogled-chromium-macos/build/src/out/Default/../../build/toolchain/apple/linker_driver.py", line 389, in <module>
LinkerDriver(sys.argv).run()
File "/Users/cubik65536/ungoogled-chromium-macos/build/src/out/Default/../../build/toolchain/apple/linker_driver.py", line 172, in run
subprocess.check_call([self._driver_path] + compiler_driver_args,
File "/opt/homebrew/Cellar/python@3.12/3.12.3/Frameworks/Python.framework/Versions/3.12/lib/python3.12/subprocess.py", line 413, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['../../third_party/llvm-build/Release+Asserts/bin/clang++', '-fuse-ld=lld', '-Wl,--icf=all', '-Wl,--color-diagnostics', '-Wl,-mllvm,-instcombine-lower-dbg-declare=0', '-Wl,-mllvm,-split-threshold-for-reg-with-hint=0', '-Wl,-cache_path_lto,thinlto-cache', '-Wl,--thinlto-cache-policy=cache_size=10%:cache_size_bytes=40g:cache_size_files=100000', '-flto=thin', '-Wl,--thinlto-jobs=all', '-Wl,-mllvm,-import-instr-limit=30', '-Wl,-mllvm,-disable-auto-upgrade-debug-info', '-fwhole-program-vtables', '-Wl,--strict-auto-link', '-Wl,--no-call-graph-profile-sort', '--target=arm64-apple-macos', '-Wl,-mllvm,-enable-machine-outliner=never', '-no-canonical-prefixes', '-Wl,-dead_strip', '-Wl,-no_data_in_code_info', '-Wl,-no_function_starts', '-nostdlib++', '-isysroot', '../../../../../../../Applications/Xcode-16.0.0-Beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX15.0.sdk', '-mmacos-version-min=10.15', '-Wl,--lto-O0', '-Wl,-ObjC', 'prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/libstd.rlib', 'prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/liballoc.rlib', 'prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/libcfg_if.rlib', 'prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/libcompiler_builtins.rlib', 'prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/libcore.rlib', 'prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/libgetopts.rlib', 'prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/libhashbrown.rlib', 'prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/liblibc.rlib', 'prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/libpanic_abort.rlib', 'prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/libpanic_unwind.rlib', 'prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/librustc_demangle.rlib', 'prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/libstd_detect.rlib', 'prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/libtest.rlib', 'prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/libunicode_width.rlib', 'prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/libunwind.rlib', 'prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/libaddr2line.rlib', 'prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/libadler.rlib', 'prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/libgimli.rlib', 'prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/libmemchr.rlib', 'prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/libminiz_oxide.rlib', 'prebuilt_rustc_sysroot/lib/rustlib/aarch64-apple-darwin/lib/libobject.rlib', '-o', './transport_security_state_generator', '@./transport_security_state_generator.rsp', 'obj/third_party/rust/cxx/v1/lib/libcxx_lib.rlib', '-Wl,-object_path_lto,tmpg5tacew5']' returned non-zero exit status 1.
[8220/60916] CXX obj/skia/skia_core_and_effects/SkFont_serial.o
ninja: build stopped: subcommand failed.
Might be beta macOS/darwin toolkit, might be other factor, I'm investigating. If someone knows some related issue and solution please help.
portable linux still fine for 126.0.6478.55
@Cubik65536 Chromium defines it's own allocator overrides in build/rust/std/remap_alloc.cc, there's some good comments about it near the top. I haven't seen any recent changes that could cause issues, so maybe there's been a change in your toolchain. I'm not familiar with your setup but you could see if the change in [5541062] helps, apparently the linking order matters. I'll see what other info I can find, but in the worst-case you could build with enable_rust=false
if we can't sort that out in time.
@Cubik65536 Chromium defines it's own allocator overrides in build/rust/std/remap_alloc.cc, there's some good comments about it near the top. I haven't seen any recent changes that could cause issues, so maybe there's been a change in your toolchain. I'm not familiar with your setup but you could see if the change in [5541062] helps, apparently the linking order matters. I'll see what other info I can find, but in the worst-case you could build with
enable_rust=false
if we can't sort that out in time.
As far as I can tell, [5541062] doesn't help. I will try to disable Rust for now and see what happends.
@Cubik65536 Chromium defines it's own allocator overrides in build/rust/std/remap_alloc.cc, there's some good comments about it near the top. I haven't seen any recent changes that could cause issues, so maybe there's been a change in your toolchain. I'm not familiar with your setup but you could see if the change in [5541062] helps, apparently the linking order matters. I'll see what other info I can find, but in the worst-case you could build with
enable_rust=false
if we can't sort that out in time.As far as I can tell, [5541062] doesn't help. I will try to disable Rust for now and see what happends.
Disable Rust causes an even larger problem. I'll try to let GitHub Action to build directly the new version and see if the problem persists...
UPD: The GitHub Action CI with the exact same environment to build older versions also have the same Rust issue.
Are you still building with the macos fix-build-with-rust.patch or at least setting rust_sysroot_absolute
via flag?
I was digging into the linking order and the only time you'd have the prebuilt rlibs as part of the ldflags is if rust_prebuilt_stdlib
is true1, which shouldn't be the case when rust_sysroot_absolute
is set2,3. Then remap_alloc wouldn't be included since toolchain_for_rust_host_build_tools
defaults to false4.
Are you still building with the macos fix-build-with-rust.patch or at least setting
rust_sysroot_absolute
via flag?
Yes, the patch is still applied, can confirm.
I was digging into the linking order and the only time you'd have the prebuilt rlibs as part of the ldflags is if
rust_prebuilt_stdlib
is true1, which shouldn't be the case whenrust_sysroot_absolute
is set2,3. Then remap_alloc wouldn't be included sincetoolchain_for_rust_host_build_tools
defaults to false4.
According to this, isn't remap_alloc
included when toolchain_for_rust_host_build_tools
is false?
And according to GitHub Action test running in my repo, only the arm64 build is failing, don't know if it is somehow related to a platform-specific issue (or maybe the x86_64 server is just too slow and the stage where it fails still haven't arrived). :(
According to this, isn't
remap_alloc
included whentoolchain_for_rust_host_build_tools
is false?
You're correct, i had that inverted. If it's only failing for the arm64 build then that at least can narrow the search some. I'll spend some more time looking into it in the morning.
According to this, isn't
remap_alloc
included whentoolchain_for_rust_host_build_tools
is false?You're correct, i had that inverted. If it's only failing for the arm64 build then that at least can narrow the search some. I'll spend some more time looking into it in the morning.
I will also try to do some more search later... Can basically confirm that only arm64 is failing...
So .61 is out, would you prefer to make that a separate PR or update this one?
I haven't made any headway with the mac arm64 build, so I'm not sure how long that will take to get sorted out.
So .61 is out, would you prefer to make that a separate PR or update this one?
I haven't made any headway with the mac arm64 build, so I'm not sure how long that will take to get sorted out.
I also wasn't able to make much progress on macOS arm64 build, maybe we could roll out .55 now and I'll skip it at the moment and try to get .61 out.
@PF4Public would you be fine with release this now as well?
@PF4Public would you be fine with release this now as well?
Absolutely! Especially now that @Cubik65536 skips .55
Just noticed. Does anyone experience protection faults upon exit (on Linux)?
[28610:28620:0615/122550.068439:ERROR:ev_root_ca_metadata.cc(162)] Failed to decode OID: 0
Speicherzugriffsfehler
traps: chrome[28610] general protection fault ip:55e03dbfaded sp:7fffe831d730 error:0 in chrome[55e03ca87000+bbaa000]
@PF4Public : I get the first error you mentioned (for quite some time i think:
[315030:315040:0615/133558.196609:ERROR:ev_root_ca_metadata.cc(162)] Failed to decode OID: 0
but not the protection fault upon exit.
I'm on manjaro by the way.
(for quite some time i think
Yes, that one is unrelated, I just gave it there for the context, otherwise leaving only protection fault would be dubious (let alone it is in German :) )
@PF4Public I haven't encountered that on my builds, but there's been some significant changes to the partition allocator and that could be a knock-on effect. You'd have to make a build without striping symbols and see if a debugger gives you any clues.
@Cubik65536 I managed to work around the linking issue by adding use_thin_lto=false
to the flags for the arm64 build. Due to the lack of other reports I suspect there might be a change in newer llvm builds that avoids this problem. If you get the time you could try building with a newer version of llvm to see if that's the case.
@PF4Public I haven't encountered that on my builds, but there's been some significant changes to the partition allocator and that could be a knock-on effect. You'd have to make a build without striping symbols and see if a debugger gives you any clues.
@Cubik65536 I managed to work around the linking issue by adding
use_thin_lto=false
to the flags for the arm64 build. Due to the lack of other reports I suspect there might be a change in newer llvm builds that avoids this problem. If you get the time you could try building with a newer version of llvm to see if that's the case.
I will try to build with the newest LLVM and also try use_thin_lto=false
(if issue persists).
Since the ungoogled patches for Chromium126.0.6478.55, I have not been able to compile the ungoogled binaries. The regular Chromium browser still compiles fine. I end up with this error:
[851/60623] STAMP obj/chrome/browser/resources/browsing_topics/preprocess_static
_files.stamp
[852/60623] LINK ./chrome_sandbox
FAILED: chrome_sandbox
"python3" "../../build/toolchain/gcc_link_wrapper.py" --output="./chrome_sandbox
" -- ../../third_party/llvm-build/Release+Asserts/bin/clang++ -pie -fuse-ld=lld
-Wl,--build-id=sha1 -fPIC -Wl,-z,noexecstack -Wl,-z,relro -Wl,-z,now -Wl,--icf=a
ll -Wl,--color-diagnostics -Wl,-mllvm,-instcombine-lower-dbg-declare=0 -Wl,-mllv
m,-split-threshold-for-reg-with-hint=0 -Wl,--thinlto-cache-dir=thinlto-cache -Wl
,--thinlto-cache-policy=cache_size=10\%:cache_size_bytes=40g:cache_size_files=10
0000 -flto=thin -Wl,--thinlto-jobs=all -Wl,-mllvm,-import-instr-limit=30 -Wl,-ml
lvm,-disable-auto-upgrade-debug-info -fwhole-program-vtables -Wl,--undefined-ver
sion -Wl,--no-call-graph-profile-sort -m64 -no-canonical-prefixes -Wl,-O2 -Wl,--
gc-sections -Wl,-z,defs -Wl,--as-needed -nostdlib++ --sysroot=../../build/linux/
debian_bullseye_amd64-sysroot -Wl,--lto-O0 -fsanitize=cfi-vcall -fsanitize=cfi-i
call -rdynamic -pie -Wl,--disable-new-dtags -o "./chrome_sandbox" -Wl,--start-gr
oup @"./chrome_sandbox.rsp" -Wl,--end-group ../../third_party/llvm-build/Relea
se+Asserts/lib/clang/19/lib/x86_64-unknown-linux-gnu/libclang_rt.builtins.a -ldl
-lpthread -lrt
ld.lld: error: cannot open Scrt1.o: No such file or directory
ld.lld: error: cannot open crti.o: No such file or directory
ld.lld: error: cannot open crtbeginS.o: No such file or directory
ld.lld: error: unable to find library -ldl
ld.lld: error: unable to find library -lpthread
ld.lld: error: unable to find library -lrt
ld.lld: error: unable to find library -lgcc_s
ld.lld: error: unable to find library -lgcc
ld.lld: error: unable to find library -lgcc_s
ld.lld: error: unable to find library -lgcc
ld.lld: error: cannot open crtendS.o: No such file or directory
ld.lld: error: cannot open crtn.o: No such file or directory
clang++: error: linker command failed with exit code 1 (use -v to see invocation
)
I build Chromium on Slackware using Google's clang and rust compilers which I download, using the Debian sysroot which I used to download and is now part of the Chromium source tarball, and with 'use_custom_libcxx=true'
.
Any ideas?
Files in the debian sysroot are being pruned, so you have to replace it with a clean copy from the archive/clone before building.
Files in the debian sysroot are being pruned, so you have to replace it with a clean copy from the archive/clone before building.
Yep that seems to have resolved my issue, thanks!
@PF4Public I haven't encountered that on my builds, but there's been some significant changes to the partition allocator and that could be a knock-on effect. You'd have to make a build without striping symbols and see if a debugger gives you any clues.
@Cubik65536 I managed to work around the linking issue by adding
use_thin_lto=false
to the flags for the arm64 build. Due to the lack of other reports I suspect there might be a change in newer llvm builds that avoids this problem. If you get the time you could try building with a newer version of llvm to see if that's the case.
The issue persisted while building with the latest LLVM, however adding the flag resolves the issue. I am building .55 locally just for testing purpose, and I will just go publish .61 once the corresponding PR is merged.
I'm all out of ideas for the moment. I'm not sure what they changed that would cause lto to not include those symbols only on arm64. I tested marking them as used with these changes which worked, but I had only built transport_security_state_generator. You could test with that change but I'd be very surprised if that doesn't result in duplicate symbol errors for some other part of the build.
I'm all out of ideas for the moment. I'm not sure what they changed that would cause lto to not include those symbols only on arm64. I tested marking them as used with these changes which worked, but I had only built transport_security_state_generator. You could test with that change but I'd be very surprised if that doesn't result in duplicate symbol errors for some other part of the build.
https://github.com/ungoogled-software/ungoogled-chromium-macos/pull/171 is up for .61, however I didn't try those changes yet, will try it later.
This PR updates to Chromium 126.0.6478.55.
Notes:
New patches:
The building with prunned binaries patch has new additions for the AI changes in [5518819] and [5431113]
The safebrowsing patch had some turmoil due changing how we deal with the enterprise connectors in response to [5485006]. There may be some platform-specific sections I missed, so let me know if you encounter any build issues.
The remove unused prefs patch had some new changes for [5446906], [5371929], [5494404], and [5377876]
0018-disable-first-run-behaviour.patch has been removed due to [5501043]
A small change for the close confirmation patch due to [5517669]. The reset was removed since it was causing crashes. Even though the original code still does the same thing, this no longer seems to be affected by the bug that fix was for.
The disable capture all screens patch had some changes due to [5428620]
The remove ui patch had an addition for the safety hub transition as well as a removal for the menu help bubble on new profiles
js_code_coverage was added to the contingent paths due to a filename difference between clone and tarball
Debian sysroots are now included in DEPS and they plan to remove the download script eventually. This means that some sysroots will now be included in the tarballs. I've opted to just allow the pruning script to do it's thing, but what this means is that anyone building with a sysroot would need to replace the pruned directory with one from a clean clone or tarball, otherwise you'll encounter missing libraries during build. Hopefully the increased size doesn't require us to raise our CI requirements...
Change log Blog post