search

OSSystems / meta-browser

OpenEmbedded/Yocto BSP layer for Web Browsers
MIT License
184 stars 191 forks source link

chromium: update to 112.0.5615.165 #709

Closed nrpt-m closed 1 year ago

nrpt-m commented 1 year ago

Release notes: https://chromereleases.googleblog.com/2023/04/stable-channel-update-for-desktop.html (112.0.5615.49) https://chromereleases.googleblog.com/2023/04/stable-channel-update-for-desktop_14.html (112.0.5615.121) https://chromereleases.googleblog.com/2023/04/stable-channel-update-for-desktop_18.html (112.0.5615.165)

Build and patch changes:

Renamed & added 0017-Revert-Fix-std-span-autodetection-7805.patch to enable building with clang 12.

Renamed & added 0018-Only-default-operator-on-declaration.patch to enable building with clang12, which doesn't implement P2085R0 and therefore requires all defaulted operator== implementations to be defined on the first declaration.

Moved fix-libc-version-include.patch to musl.

Rebase remaining patches.

License changes:

Added licenses:

Removed licenses:

Updated licenses:

Test-built:

** Please note that Chromium requires below set-up when on dunfell branch.

Signed-off-by: Narpat Mali narpat.mali@windriver.com

nrpt-m commented 1 year ago

@MaxIhlenfeldt, @rakuco, After updating to 112.0.5615.165 , have successfully built for master branch and all systems worked well. Have also built for dunfell branch & observed that build for all systems completed successfully except qemuarm64.

Have got this below error while building for qemuarm64:

ERROR: chromium-x11-112.0.5615.165-r0 do_update_node_modules: Execution of '/buildarea/eng1/chromium/b/build-dunfell/tmp/work/aarch64-poky-linux/chromium-x11/112.0.5615.165-r0/temp/run.do_update_node_modules.3507160' failed with exit code 1 DEBUG: Executing shell function do_update_node_modules npm WARN webui-node-modules@1.0.0 No repository field. npm WARN webui-node-modules@1.0.0 No license field. npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@2.3.2 (node_modules/fsevents): npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@2.3.2: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
audited 258 packages in 7.306s
42 packages are looking for funding
run npm fund for details
found 0 vulnerabilities
patching file index.d.ts
Reversed (or previously applied) patch detected! Assume -R? [n]
Apply anyway? [n]
Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file index.d.ts.rej
WARNING: /buildarea/eng1/chromium/b/build-dunfell/tmp/work/aarch64-poky-linux/chromium-x11/112.0.5615.165-r0/temp/run.do_update_node_modules.3507160:1 exit 1 from 'patch -d node_modules/@types/d3/ -p1 < chromium_d3_types_index.patch'

Any suggestion ?

nrpt-m commented 1 year ago

Thanks for the update. The nodejs error sounds very weird and I'm not sure how to fix it yet.

In the meantime, could you just update the commit/PR message and include what's in #706 (i.e. the license changes and the rest), and mention @MaxIhlenfeldt and @kraj as contributors?

Updated all the commits.

nrpt-m commented 1 year ago

@rakuco , @MaxIhlenfeldt , Have also built & tested on kirkstone & mickledore branches and below were the observations. For kirkstone, all systems work well. On qemux86-64, have observed this below error in kernel log as well as in chromium execution logs but, chromium works fine.

libGL error: pci id for fd 4: 1234:1111, driver (null) ibGL error: MESA-LOADER: failed to open bochs-drm: /usr/lib/dri/bochs-drm_dri.so: cannot open shared object file: No such file or directory (search paths /usr/lib/dri, suffix _dri) libGL error: failed to load driver: bochs-drm

For mickledore, all systems work well except qemuarm. It is using the 6.1.25 kernel on qemuarm and we see the following error:

vmap allocation for size 4100096 failed: use vmalloc= to increase size

System appears to boot well but, user-interface is not responding. Earlier, this issue was not observed with the use of Linux-yocto-dev kernel 6.3.0 version but now, with this kernel as well we observed same.

kraj commented 1 year ago

@rakuco , @MaxIhlenfeldt , Have also built & tested on kirkstone & mickledore branches and below were the observations. For kirkstone, all systems work well. On qemux86-64, have observed this below error in kernel log as well as in chromium execution logs but, chromium works fine.

libGL error: pci id for fd 4: 1234:1111, driver (null) ibGL error: MESA-LOADER: failed to open bochs-drm: /usr/lib/dri/bochs-drm_dri.so: cannot open shared object file: No such file or directory (search paths /usr/lib/dri, suffix _dri) libGL error: failed to load driver: bochs-drm

For mickledore, all systems work well except qemuarm. It is using the 6.1.25 kernel on qemuarm and we see the following error:

vmap allocation for size 4100096 failed: use vmalloc= to increase size

this is perhaps fixed with https://lists.openembedded.org/g/openembedded-core/message/180198

System appears to boot well but, user-interface is not responding. Earlier, this issue was not observed with the use of Linux-yocto-dev kernel 6.3.0 version but now, with this kernel as well we observed same.

kraj commented 1 year ago

@rakuco , @MaxIhlenfeldt , Have also built & tested on kirkstone & mickledore branches and below were the observations. For kirkstone, all systems work well. On qemux86-64, have observed this below error in kernel log as well as in chromium execution logs but, chromium works fine.

libGL error: pci id for fd 4: 1234:1111, driver (null) ibGL error: MESA-LOADER: failed to open bochs-drm: /usr/lib/dri/bochs-drm_dri.so: cannot open shared object file: No such file or directory (search paths /usr/lib/dri, suffix _dri) libGL error: failed to load driver: bochs-drm

I think this is perhaps harmless.

For mickledore, all systems work well except qemuarm. It is using the 6.1.25 kernel on qemuarm and we see the following error:

vmap allocation for size 4100096 failed: use vmalloc= to increase size

System appears to boot well but, user-interface is not responding. Earlier, this issue was not observed with the use of Linux-yocto-dev kernel 6.3.0 version but now, with this kernel as well we observed same.

rwmacleod commented 1 year ago

@khem, I've asked Ross and Steve if the "machine/qemuarm*: don't explicitly set vmalloc " commit can / should be back-ported to mickledore.

@MaxIhlenfeldt @nrpt-m has seen two problems:

  1. qeumarm64 build fails on dunfell. Can you reproduce that?
  2. On dunfell, if you build once, it will succeed for qemuarm for example, then the qemuarm64 build failed and Narpat tried to build a second time for qemuarm, without doing a clean, that build failed as well. I suspect but we haven't yet confirmed that any second build will fail because the new stage added always attempt to re-apply the patches. Narpat to confirm.

More to come tomorrow.

nrpt-m commented 1 year ago
  1. On dunfell, if you build once, it will succeed for qemuarm for example, then the qemuarm64 build failed and Narpat tried to build a second time for qemuarm, without doing a clean, that build failed as well. I suspect but we haven't yet confirmed that any second build will fail because the new stage added always attempt to re-apply the patches. Narpat to confirm.

More to come tomorrow.

Have confirmed that "any second build is not failing" by building qemuarm on dunfell 2-3 times & it completed without any error.

nrpt-m commented 1 year ago

@kraj I think we are close but, we can't access the network outside the fetch stage. @rwmacleod and I concluded that rather then running npm after patching we could run npm once manually and then generated a patch & save it. Then, we can take @MaxIhlenfeldt patches along with our saved patch and apply them all during the do_patch stage.

We have done that by using npm from the host and unfortunately the patch is 44MB ! We are going to try using the version of npm that shipped with dunfell to see if it generates a smaller patch.

Does it seems like good idea ? Would the 44MB patch be acceptable ? Currently the meta-browser layer is 14MB. chromium-112.0.5615.165-dunfell-rollup-3.12.0-2.58.0-diffstat.txt

ekapllaj commented 1 year ago

@nrpt-m you can enable the network on a particular task this way (for example on compile task): do_compile[network] = "1"

rwmacleod commented 1 year ago

@ekapllaj yes, one can but that's not a good idea and some OE users need to have the build system and all related source delivered to them so they can do offline builds. That seems kind of outrageous when you're building the chromium browser but those are the requirements that certain organizations have. Given that patching the source code is equivalent to running the 'npm install' command, I think the cost of the rather large 44 MB patch is worthwhile.

That said I do need to become more familiar with npm packaging in Yocto. I acknowledged that the source for the rollup module is < 2 MB and that ideally, we'd figure out how to get npm to fetch from the local source caches rather than accessing the internet. We'd still need to do the fetch during the fetch stage.

otavio commented 1 year ago

I would like to focus on discussing offline building further and exploring other options. The size of the 44MB patch is unreasonable, particularly considering that it will vary with each new release.

rwmacleod commented 1 year ago

@otavio sounds good.

I came to that conclusion about an hour after @nrpt-m and I coughed up the 44 MB patch! ;-)

I'll continue to work on it with @nrpt-m for an hour or two a day unless someone else picks up the ball full-time. Some background is here: https://wiki.yoctoproject.org/wiki/TipsAndTricks/NPM and the various commit logs and some YP documentation may even exist.

MaxIhlenfeldt commented 1 year ago

Hi all, I still don't have a lot of time for this. I fear it might continue to be like this until W20 (i.e. still no time next week), sorry for that.

Regarding the NPM stuff, do we actually have to run NPM to replace the version of rollup used? Maybe it's enough to ship https://registry.npmjs.org/rollup/-/rollup-2.58.0.tgz, which is only 1.1 MB, and extract it over the existing dist? I have no idea if that might lead to any dependency problems, but it may be worth a try.

rakuco commented 1 year ago

There's also the possibility of adding a new nodejs14 recipe to dunfell as discussed here: https://github.com/OSSystems/meta-browser/pull/706#issuecomment-1518890518

I'm not sure how much work that would be (i.e. how many different architectures need to be tested and what tests to run besides just building the recipe), but it would be the cleanest solution for us, as I expect Chromium to increasingly depend on new rollup features over time.

MaxIhlenfeldt commented 1 year ago

That would be great of course. But I suppose it would take some time until that's usable, and we probably need a solution for the mean time?

rakuco commented 1 year ago

@kraj could you give more details on what kind of testing would be needed for this recipe to be added? Looking at node 14's release notes I don't see many/any breaking changes, in the best case scenario copy-pasting the node recipe from a more recent branch would Just Work.

nrpt-m commented 1 year ago

Hi all, Have updated the nodejs_12.22.12 recipe to nodejs-native_14.18.1 and commented the "0017-Revert-WebUI-Update-Rollup-to-v3.12.0.patch" along with the workaround changes for dunfell in the chromium recipe.

With these above changes, we are able to build the qemux86-64 & rpi4 for dunfell without any error/issue. Now, the build is using nodejs-native_14.18.1, while building the chromium 112.0.5615.165 version. Will build the remaining systems and test.

@rwmacleod Please add, if I have missed anything.

rwmacleod commented 1 year ago

@nrpt-m Good summary. We'll work on some PREFERRED_VERSION changes tomorrow ensure that nodejs_12 is the default for normal uses of meta-oe but that chromium requires and depends on nodejs-native_14.

kraj commented 1 year ago

Hi all, Have updated the nodejs_12.22.12 recipe to nodejs-native_14.18.1 and commented the "0017-Revert-WebUI-Update-Rollup-to-v3.12.0.patch" along with the workaround changes for dunfell in the chromium recipe.

With these above changes, we are able to build the qemux86-64 & rpi4 for dunfell without any error/issue. Now, the build is using nodejs-native_14.18.1, while building the chromium 112.0.5615.165 version. Will build the remaining systems and test.

@rwmacleod Please add, if I have missed anything.

Can you post the nodejs upgrade patch to dunfell mega-oe

nrpt-m commented 1 year ago

@kraj , The patch is not quite ready.

@rwmacleod tells me that he is little rusty about PREFERRED_VERSION settings but, we tried making the following change:

meta-oe/conf/layer.conf +PREFERRED_VERSION_nodejs ?= "12.%" +PREFERRED_VERSION_nodejs-native ?= "14.%"

We were hoping that the native version of nodejs would not matter but, unfortunately, bitbake nodejs(12) fails with the error: ../../deps/v8/src/builtins/array-join.tq:151:8: Torque Error: Parser Error: unexpected token "("

This above error comes from the torque executable running on this tq file. It appears like syntax changed between version 12 and 14. :-(

We are trying not to change nodejs_12 behavior while making nodejs-native_14 available to chromium. Chromium recipe is used in many branches so, without making a branch in meta-browser for dunfell, we don't see how to introduce a dependency on nodejs-native_14 using either a PREFERRED_VERSION or by calling the recipe nodejs-14-native since, that will force all branches to use nodejs-14-native even if they are not using the dunfell branch.

Is there a way to achieve this goal without introducing a branch in meta-browser for dunfell ?

As a test, we have added PREFERRED_VERSION_nodejs-native = "14.%" to conf/local.conf and built bitbake chromium-x11. That picks up nodejs-native_14 as expected and the build completed successfully.

Other solution is that we would required to add PREFERRED_VERSION_nodejs-native = "14.%" to conf/local.conf file while building the chromium for dunfell. More to come tomorrow...

ekapllaj commented 1 year ago

Hi, I actually would prefer the creation of a branch for dunfell. The dunfell distro is going to be around at least until next year (Apr 2024), and I think this issue is comming to bite as at every update. I'm building on Honister and chromium 113 is not compiling at all, so I think it's going to be complicated to try to compile for dunfell for next releases.

I would say that the best compromise would be to kind of freeze the chromium 112 on a separate branch for dunfell (with the appropriate patches) and move on on master branch.

otavio commented 1 year ago

If we choose the dunfell branch, we can branch out on 111 and implement the necessary build fixes. I'm okay with that, but everyone needs to understand that no security updates will be applied.

rakuco commented 1 year ago

I'm in the "those who do the work decide" camp. I've been happily reviewing pull requests but haven't worked on updates myself for a while. If the people who have been carrying out the work like @MaxIhlenfeldt, @rwmacleod and @nrpt-m feel like it's no longer worth supporting dunfell (due to either the maintenance burden or lack of client demand), I certainly have no objections.

dunfell users might not be very happy, but it would certainly make updates a lot easier for us.

nrpt-m commented 1 year ago

Hi All,

We had sent the patch to make nodejs_14.18.1 available but not default using DEFAULT_PREFERENCE to make sure that the default version of nodejs remains 12.x. Patch link: https://lore.kernel.org/openembedded-devel/20230511163758.358936-1-narpat.mali@windriver.com/

It's on the way to get it merge into dunfell: https://lore.kernel.org/openembedded-devel/095ad873-2d15-5f66-52fc-5c4e34be3348@gmail.com/

otavio commented 1 year ago

Are you looking to include a note regarding the usage of the dunfell branch that mentions its restriction?

nrpt-m commented 1 year ago

Are you looking to include a note regarding the usage of the dunfell branch that mentions its restriction?

@otavio Have already added the set-up requirements when on dunfell in the PR & commit messages.

kraj commented 1 year ago

Patch to add nodejs 14 has landed in meta-openembedded/dunfell https://git.openembedded.org/meta-openembedded/commit/?h=dunfell&id=116bfe8d5e5851e7fc5424f40da8691a19c5b5ee

rwmacleod commented 1 year ago

Looks like we're finally good to go. @MaxIhlenfeldt @rafluan @kraj please merge if you agree.

kraj commented 1 year ago

@nrpt-m Can you address these comments and upload again

nrpt-m commented 1 year ago

Thanks a lot for finishing this! Almost LGTM, just some polishing comments.

Can you please remove the mentions of 0017-Revert-WebUI-Update-Rollup-to-v3.12.0.patch and 0019-Backport-Fix-deps-for-feed_response.proto.patch from the PR description? Not sure if it makes sense to also squash the commits, so that the new single commit message matches the description?

Have removed the mentions of 0017-Revert-WebUI-Update-Rollup-to-v3.12.0.patch and 0019-Backport-Fix-deps-for-feed_response.proto.patch from the PR description but, kept as it is in commit message.

nrpt-m commented 1 year ago

@nrpt-m Can you address these comments and upload again

Done. Shall I move "fix-libc-version-include.patch" patch into musl/ as @MaxIhlenfeldt suggested ?

kraj commented 1 year ago

@nrpt-m Can you address these comments and upload again

Done. Shall I move "fix-libc-version-include.patch" patch into musl/ as @MaxIhlenfeldt suggested ?

yes

nrpt-m commented 1 year ago

@MaxIhlenfeldt , @kraj Requested changes has been done and let me know if any further modification needed.