Update to version 65

[Eloston]

This issue tracks the upgrade progress to Chromium 65, subsequent revisions, and related discussion.

Here are the steps necessary towards the first release in the upgrade process (in approximate order):

• [x] Include new patches from Debian
• [x] Include new patches from Inox
• [x] Include (or upgrade) patches from Iridium
• [x] Update ungoogled-chromium's own patches
• [x] Informal testing (e.g. checking for major regressions, testing new features)
• [x] Publish new release and binaries

After the initial release is published, this issue will remain open for general discussions about upgrading platforms that are not working in this new version. Specific issues should be posted in a separate issue.

[Eloston]

Thanks to @squalus and @xsmile, most (if not all) of the work for 65 is already done. I'll be making a build for Debian and possibly tweak a few minor things.

[Eloston]

I am running a new 65 build on Debian 9 amd64, but there is something wrong with VP9 decoding (but not VP8 or H264) with VA-API hardware acceleration on a Kabylake machine versus the Skylake build machine. I'm not sure if it's a Mesa problem or the use of the bundled libvpx library (since the one in Debian 9 would need some patching). If anyone has some insights on this problem, I'd like to know.

EDIT: To elaborate more visually, the video appears as a garbled mess that is partitioned horizontally into 2 or 4 (or possibly more) pieces. Though on occasion, the video will render normally for a short time.

[xsmile]

If the issue only affects VA-API, it should be related to the libva library. If I'm not mistaken Debian 9 uses libva1, while libva2 has been out for some time including several Kabylake and VP9 fixes. I'm not sure why the issue appeared with 65 though.

[Eloston]

@xsmile Yeah that seems to be it. Disabling hardware video decode works around the problem.

[tectiv3]

macOS build built successfully. Going to test it for a few days. Binary uploaded here

[tectiv3]

So far so good, the only thing I've noticed is a high CPU usage in developer tools Inspect element and a few artefacts:

[Eloston]

@tectiv3 I'm not getting any of those problems in my Debian build. Those artifacts appear to be some low-level rendering issue in Chromium (for macOS) or a bug in macOS.

[arutr]

@tectiv3 Could you provide steps to reproduce the artifacts?

[Eloston]

Here are my Debian 9 amd64 binaries of 65.0.3325.162-1. Other than the aforementioned Kabylake VA-API VP9 decoding issue, I haven't encountered any other issues yet.

[tectiv3]

@Artur96 just open up developer tools, it's right there. Doesn't bother me much, just thought I should mention it.

[arutr]

@tectiv3 I don't get any artifacts. What macOS version do you have?

[tectiv3]

@Artur96 10.11

[Eloston]

@tectiv3 @Artur96 Are you guys running the same binaries? If not, you could consider sending each other's builds. Then we can determine if the problem is due to the build environment or the runtime environment.

[arutr]

@Eloston I've built on my own machine (10.13.3).

[tectiv3]

@Artur96 can you please try to run my build? (https://github.com/tectiv3/ungoogled-chromium-binaries/releases/tag/65.0.3325.162-1)

[arutr]

@tectiv3 No issues with your build either

[tectiv3]

@Artur96 thank you. Then it's the OS.

[nikolowry]

Building and running fine on Arch (2+ days as daily driver). However, I've been struggling with trying to customize the build to support Vulkan (enable_vulkan=true) -- if anyone has some pointers, don't hesitate to share.

Currently, the build errors out nearly at the halfway mark with the following:

[8624/21757] CXX obj/skia/skia/SkImage.o
FAILED: obj/skia/skia/SkImage.o 
clang++ -MMD -MF obj/skia/skia/SkImage.o.d -DV8_DEPRECATION_WARNINGS -DUSE_UDEV -DUSE_AURA=1 -DUSE_GLIB=1 -DUSE_NSS_CERTS=1 -DUSE_X11=1 -DNO_TCMALLOC -DCHROMIUM_BUILD -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -DNO_UNWIND_TABLES -DCR_CLANG_REVISION=\"321529-2\" -DNDEBUG -DNVALGRIND -DDYNAMIC_ANNOTATIONS_ENABLED=0 -DSK_IGNORE_LINEONLY_AA_CONVEX_PATH_OPTS -DSK_HAS_PNG_LIBRARY -DSK_HAS_WEBP_LIBRARY -DSK_HAS_JPEG_LIBRARY -DSK_VULKAN_HEADER=\"../../skia/config/SkVulkanConfig.h\" -DSK_VULKAN=1 -DSK_SUPPORT_GPU=1 -DSK_GAMMA_EXPONENT=1.2 -DSK_GAMMA_CONTRAST=0.2 -DSK_DEFAULT_FONT_CACHE_LIMIT=20971520 -DGLIB_VERSION_MAX_ALLOWED=GLIB_VERSION_2_32 -DGLIB_VERSION_MIN_REQUIRED=GLIB_VERSION_2_26 -DUSE_SYSTEM_ZLIB=1 -DUSE_SYSTEM_LIBJPEG -DUSING_SYSTEM_ICU=1 -DICU_UTIL_DATA_IMPL=ICU_UTIL_DATA_STATIC -DUCHAR_TYPE=uint16_t -I../.. -Igen -I../../skia/config -I../../skia/ext -I../../third_party/skia/include/c -I../../third_party/skia/include/config -I../../third_party/skia/include/core -I../../third_party/skia/include/effects -I../../third_party/skia/include/encode -I../../third_party/skia/include/gpu -I../../third_party/skia/include/images -I../../third_party/skia/include/lazy -I../../third_party/skia/include/pathops -I../../third_party/skia/include/pdf -I../../third_party/skia/include/pipe -I../../third_party/skia/include/ports -I../../third_party/skia/include/utils -I../../third_party/vulkan/include -I/usr/include -I../../third_party/skia/src/gpu -I../../third_party/skia/src/sksl -I../../third_party/skia/include/gpu/vk -I../../third_party/skia/include/codec -I../../third_party/skia/include/private -I../../third_party/skia/include/client/android -I../../third_party/skia/src/codec -I../../third_party/skia/src/core -I../../third_party/skia/src/image -I../../third_party/skia/src/images -I../../third_party/skia/src/opts -I../../third_party/skia/src/pdf -I../../third_party/skia/src/ports -I../../third_party/skia/src/shaders -I../../third_party/skia/src/shaders/gradients -I../../third_party/skia/src/sfnt -I../../third_party/skia/src/utils -I../../third_party/skia/src/lazy -I../../third_party/skia/third_party/gif -I../../third_party/skia/src/effects/gradients -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -Igen/shim_headers/libevent_shim -Igen/shim_headers/zlib_shim -Igen/shim_headers/icuuc_shim -I../../third_party/libpng -I/usr/include/freetype2 -I/usr/include/libpng16 -I/usr/include/harfbuzz -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -I../../third_party/sfntly/src/cpp/src -fno-strict-aliasing --param=ssp-buffer-size=4 -fstack-protector -Wno-builtin-macro-redefined -D__DATE__= -D__TIME__= -D__TIMESTAMP__= -fno-unwind-tables -fno-asynchronous-unwind-tables -fPIC -pipe -pthread -fcolor-diagnostics -no-canonical-prefixes -m64 -march=x86-64 -fno-omit-frame-pointer -g0 -fvisibility=hidden -Wall -Wno-unused-variable -Wno-missing-field-initializers -Wno-unused-parameter -Wno-c++11-narrowing -Wno-covered-switch-default -Wno-unneeded-internal-declaration -Wno-inconsistent-missing-override -Wno-undefined-var-template -Wno-nonportable-include-path -Wno-address-of-packed-member -Wno-unused-lambda-capture -Wno-user-defined-warnings -Wtautological-constant-out-of-range-compare -O2 -fno-ident -fdata-sections -ffunction-sections -std=gnu++14 -fno-exceptions -fno-rtti -fvisibility-inlines-hidden -D_FORTIFY_SOURCE=2 -march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong -fno-plt -c ../../third_party/skia/src/image/SkImage.cpp -o obj/skia/skia/SkImage.o
In file included from ../../third_party/skia/src/image/SkImage.cpp:29:
In file included from ../../third_party/skia/include/gpu/GrTexture.h:12:
In file included from ../../third_party/skia/include/gpu/GrBackendSurface.h:16:
In file included from ../../third_party/skia/include/gpu/vk/GrVkTypes.h:13:
../../third_party/skia/include/gpu/vk/GrVkDefines.h:51:2: error: "Vulkan header version is too low"
#error "Vulkan header version is too low"

Attempting a build now that omits the patches to determine whether it's a build configuration error on my part or if a patch is the culprit. If it turns out to be a build configuraiton issue, I'm planning on copying some of the GN args' values from https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=chromium-dev&id=8670e9e1691649f1424759b7b258572bdc6165ef.

I'll comment back here either way with my findings, and possibly open a new issue once the root cause of the error is.

---

I've made no progress tracking the root of this error, going to pause on my trial/error attempts until v66

[Eloston]

@nikolowry Just in case you missed it, did you see lines 431-435 in that PKGBUILD you referenced? Also, I'm guessing you might need to ensure the Vulkan code (lines 254, 265, and 266 in that PKGBUILD) are kept (though I don't think our PKGBUILD would remove it by default anyway).

[dubvulture]

Successfully built on Ubuntu 16.04. link Too early to comment on #362

[nikolowry]

@Eloston tried a bunch of different ways (even compiled Clang locally) -- but no luck.

After thinking about it I don't know if it was the Vulkan or SwiftShader support that fixes my specific GPU problems (blinking mouse pointer when hovering over GTK3 widgets) when running a mixed dpi multi monitor setup (i7 Skylake laptop w Intel HD).

I'll keep experimenting, the inox-dev/beta builds seemed to have figured it out.

[Eloston]

@nikolowry I see. In that case, waiting for v66 may not be a bad idea.

[Eloston]

Here are my Debian 9 amd64 65.0.3325.181-1 binaries. Built at commit fc3e102bba0b80acba7ce2203f3fd7527f45aaff

[Eloston]

Status update: It seems like platform support is pretty stable across the board now, so I'm planning on making the initial release of 65 within the next day or two.

Between now and then, I will be making a build of linux_portable with the linux_simple packaging type to update the stale "Semi-statically-linked Linux build" (which will be renamed to "Portable Linux build"). If it goes well, I will publish the build along with the release.

I would also like to see the release of a new Windows build this time around. If you can make a build on a clean Windows system, please let us know here. Alternatively, getting a build on an online build service may work as well.

If there are any minor fixes you wish to make, please submit them now. You may submit larger changes, but I won't merge them until after the initial release.

Also, I'd like to say thanks to everyone for their help and support thus far. It's been nice to see activity grow over time.

[Eloston]

Portable linux build is available here (as the ungoogled-chromium_65.0.3325.181-1_linux.tar.xz file).

[Eloston]

65.0.3325.181-1 has been released. I'll be uploading my Debian 9 amd64 and Portable Linux builds to the contributor binaries page.

I'll accept PRs to the downloads page for macOS and Windows binaries. The builds should be made at earliest 5d056b3da816b024cc3597419e20241caecc9ad6.

[squalus]

Windows binaries available here: https://github.com/squalus/ungoogled-chromium-winbuild/releases/tag/65.0.3325.181-1

This was done on the build system available in the same repo

[elaine-jackson]

@Eloston I have a working macOS Build, is there a way I can share with you for a public release binary?

[Eloston]

@nsuchy Yes, the instructions are linked from the README. Should I add an additional link to the contributing section?

[elaine-jackson]

@Eloston Thanks for the quick response, it's late in my timezone, in the morning I will read the instructions and upload my personal build and submit a PR.

[Eloston]

@squalus I wonder if the family of issues #380, #381, and #382 are related to is_multi_dll_chrome=false (or even use_jumbo_build=true). Maybe we could try disabling the trk scheme code net-add-trk-scheme-and-help-identify-URLs-being-retr.patch (and not apply intercept-all-modified-domains.patch and windows-fix-non-multi-dll-build.patch).

If is_multi_dll_chrome=false is the cause, I'll look into rewriting the trk: scheme code to work with multiple DLLs.

[squalus]

@Eloston This sounds plausible. Another culprit might be the windows-fix-building-without-safebrowsing.patch where some of the prefs no longer get registered. There was one crasher related to that. Regardless I will try to get more information from a debug build

[Eloston]

@squalus Awesome. Looking forward to it.

Also, it seems Iridium ran into the same Incognito issue, but I can't find any changes in their Git repo (their only location of updated source code) for this; the latest changes are from 2017-11-30. Also, the download page for Windows is at version 2017.11-1, which doesn't exist in their Git repository. The GitHub issue: https://github.com/iridium-browser/tracker/issues/184

On a tangential note, I wonder what is happening to Iridium. I can't find a link to their mailing list anymore (which is referenced several times throughout the site, e.g. the Contribute page), and this change to their website is intriguing me: https://github.com/iridium-browser/website/commit/42dfd3d411080d3526593b9206313ce6cbc41fe3. Also the footer of their website says:

Connect with the team and chat with us. Create an account on our Mattermost.

It seems Mattermost requires registration to do anything, which is a big downside compared to their mailing list.

I was hoping to get a lead on these problems from Iridium, but I don't like how inconvenient it has become to contact them (and how closed-up development feels now). It's quite unfortunate.

EDIT: s/froom/from/

[squalus]

How much value does the Iridium trk: feature really add? It seems that the subdom step destroys all the bad urls. For the trk: feature, urls must be manually patched to add the prefix, and this is only covering a subset of the bad urls.

It does result in the infobar showing up sometimes but that's hardly useful for a normal user. They really don't need to know about this; the request to Google didn't go through. It's more like a warning to tell developers to try to remove the code that made the request.

[Eloston]

@squalus I overloaded the trk feature to detect all requests with the domain substitution-exclusive string qjz9zk. So it's also useful to disable unnecessary services that fail to connect to Google, and potentially other related code that affects privacy and security.

EDIT: That is where the intercept-all-modified-domains.patch name comes from.

[squalus]

If is_multi_dll_chrome=false is the cause, I'll look into rewriting the trk: scheme code to work with multiple DLLs.

@Eloston A build with is_multi_dll_chrome=true works fine, so this does appear to be the cause of #380. I removed all the trk-related patches and produced a build with a chrome_child.dll.

[Eloston]

@squalus Thanks for confirming. I'll get to rewriting the trk code and let you know when it's ready.

[Eloston]

Status update: I've pushed da4ccbd269facfce83e2d34ae062b84e244dce9c, but the SEGV_MAPERR is unacceptable and I don't want to go back to Iridium's solution since it's a huge hack (but a cleverly written one).

I'm guessing the difference between Iridium's working solution (on non-multi-component builds at least) and my patch is the placement of the function invoked by URLRequest in chrome_main.cc instead of in browser_process_impl.cc like I did. But, I'm still not exactly sure why URLRequest throws these errors and not HandleTraceScheme (in browser_url_handler_impl.cc); I thought that everything is in one binary and thus should be accessable, but it seems I was wrong. Another strange issue I was having in a previous iteration of the patch was that using AttemptTrkNotification (from util.cc) caused SIGTRAP and made the whole browser close after logging my error message. If anyone knows what could be happening in these two issues, I'd like to know for future reference.

Instead, I recently discovered URLRequestInterceptor, which can intercept the creation of URLRequestJob (see the design docs for more info about the networking stack). There are quite a few implementations of URLRequestInterceptor that live close to or in the UI code, so I'm hoping to implement my own URLRequestInterceptor to create the infobars (and hopefully it will work on Windows too).

If it doesn't work, I am thinking about modifying the patch I committed to only log the errors and possibly redirect the user to a custom internal page (for the HandleTraceScheme route) to let them know about the interception. This should be pretty safe and straight-forward to implement given that logging seems to be working fine in the current patch.

@squalus @xsmile The patches and patch order have changed a bit in this commit. Feel free to update your v66 branches against these changes; I'll only be changing block-trk-and-substituted-domains-in-urlrequest.patch to solve this issue.

[squalus]

@Eloston A linux_simple build worked for me, but I got an error on the Windows build. I used da4ccbd + some changes to switch back to multi-dll and remove a remaining reference to iridium:trknotify in windows-disable-reorder-fix-linking.patch.

    default: [9196/24174] LINK transport_security_state_generator.exe
    default: FAILED: transport_security_state_generator.exe 
    default: cmd /c C:/Python27/python.exe ../../build/toolchain/win/tool_wrapper.py delete-file ./transport_security_state_generator.exe.pdb && C:/Python27/python.exe ../../build/toolchain/win/tool_wrapper.py link-wrapper environment.x64 False ../../third_party/llvm-build/Release+Asserts/bin/lld-link.exe /nologo /OUT:./transport_security_state_generator.exe /PDB:./transport_security_state_generator.exe.pdb @./transport_security_state_generator.exe.rsp
    default: ..\..\third_party\llvm-build\Release+Asserts\bin\lld-link.exe: error: obj/third_party/ungoogled/util/util.obj: undefined symbol: ?kTraceScheme@url@@3QBDB
    default: ..\..\third_party\llvm-build\Release+Asserts\bin\lld-link.exe: error: obj/third_party/ungoogled/util/util.obj: undefined symbol: ?SchemeIs@GURL@@QEBA_NV?$BasicStringPiece@V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@base@@@Z
    default: ..\..\third_party\llvm-build\Release+Asserts\bin\lld-link.exe: error: obj/third_party/ungoogled/util/util.obj: undefined symbol: ?spec@GURL@@QEBAAEBV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@XZ

[Eloston]

Since 66 is out, I decided to postpone my URLRequestImplementor redesign for 66.

@squalus I have pushed 643b6adc86f258cda2500d05feb75c51f41f07af which disables the trk infobar completely and has an attempt to fix your linker error. Also, how far along is Windows support for 66? If it's close enough to being done, I will probably merge @xsmile's and your branches into develop now that I've started the v66 development cycle (#392).

[Eloston]

Closing due to the beginning of the v66 development cycle #392.

[squalus]

@Eloston Great, I'll re-test. My Windows branch was working and ready for testing at the time I wrote it. I haven't rebased it yet though on the new patches.