Closed nrpt-m closed 1 year ago
@MaxIhlenfeldt, @rakuco, After updating to 112.0.5615.165 , have successfully built for master branch and all systems worked well. Have also built for dunfell branch & observed that build for all systems completed successfully except qemuarm64.
Have got this below error while building for qemuarm64:
ERROR: chromium-x11-112.0.5615.165-r0 do_update_node_modules: Execution of '/buildarea/eng1/chromium/b/build-dunfell/tmp/work/aarch64-poky-linux/chromium-x11/112.0.5615.165-r0/temp/run.do_update_node_modules.3507160' failed with exit code 1 DEBUG: Executing shell function do_update_node_modules npm WARN webui-node-modules@1.0.0 No repository field. npm WARN webui-node-modules@1.0.0 No license field. npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@2.3.2 (node_modules/fsevents): npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@2.3.2: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"}) audited 258 packages in 7.306s 42 packages are looking for funding run npm fund
for detailsfound 0 vulnerabilities patching file index.d.ts Reversed (or previously applied) patch detected! Assume -R? [n] Apply anyway? [n] Skipping patch. 1 out of 1 hunk ignored -- saving rejects to file index.d.ts.rej WARNING: /buildarea/eng1/chromium/b/build-dunfell/tmp/work/aarch64-poky-linux/chromium-x11/112.0.5615.165-r0/temp/run.do_update_node_modules.3507160:1 exit 1 from 'patch -d node_modules/@types/d3/ -p1 < chromium_d3_types_index.patch'
Any suggestion ?
Thanks for the update. The nodejs error sounds very weird and I'm not sure how to fix it yet.
In the meantime, could you just update the commit/PR message and include what's in #706 (i.e. the license changes and the rest), and mention @MaxIhlenfeldt and @kraj as contributors?
Updated all the commits.
@rakuco , @MaxIhlenfeldt , Have also built & tested on kirkstone & mickledore branches and below were the observations. For kirkstone, all systems work well. On qemux86-64, have observed this below error in kernel log as well as in chromium execution logs but, chromium works fine.
libGL error: pci id for fd 4: 1234:1111, driver (null) ibGL error: MESA-LOADER: failed to open bochs-drm: /usr/lib/dri/bochs-drm_dri.so: cannot open shared object file: No such file or directory (search paths /usr/lib/dri, suffix _dri) libGL error: failed to load driver: bochs-drm
For mickledore, all systems work well except qemuarm. It is using the 6.1.25 kernel on qemuarm and we see the following error:
vmap allocation for size 4100096 failed: use vmalloc= to increase size
System appears to boot well but, user-interface is not responding. Earlier, this issue was not observed with the use of Linux-yocto-dev kernel 6.3.0 version but now, with this kernel as well we observed same.
@rakuco , @MaxIhlenfeldt , Have also built & tested on kirkstone & mickledore branches and below were the observations. For kirkstone, all systems work well. On qemux86-64, have observed this below error in kernel log as well as in chromium execution logs but, chromium works fine.
libGL error: pci id for fd 4: 1234:1111, driver (null) ibGL error: MESA-LOADER: failed to open bochs-drm: /usr/lib/dri/bochs-drm_dri.so: cannot open shared object file: No such file or directory (search paths /usr/lib/dri, suffix _dri) libGL error: failed to load driver: bochs-drm
For mickledore, all systems work well except qemuarm. It is using the 6.1.25 kernel on qemuarm and we see the following error:
vmap allocation for size 4100096 failed: use vmalloc= to increase size
this is perhaps fixed with https://lists.openembedded.org/g/openembedded-core/message/180198
System appears to boot well but, user-interface is not responding. Earlier, this issue was not observed with the use of Linux-yocto-dev kernel 6.3.0 version but now, with this kernel as well we observed same.
@rakuco , @MaxIhlenfeldt , Have also built & tested on kirkstone & mickledore branches and below were the observations. For kirkstone, all systems work well. On qemux86-64, have observed this below error in kernel log as well as in chromium execution logs but, chromium works fine.
libGL error: pci id for fd 4: 1234:1111, driver (null) ibGL error: MESA-LOADER: failed to open bochs-drm: /usr/lib/dri/bochs-drm_dri.so: cannot open shared object file: No such file or directory (search paths /usr/lib/dri, suffix _dri) libGL error: failed to load driver: bochs-drm
I think this is perhaps harmless.
For mickledore, all systems work well except qemuarm. It is using the 6.1.25 kernel on qemuarm and we see the following error:
vmap allocation for size 4100096 failed: use vmalloc= to increase size
System appears to boot well but, user-interface is not responding. Earlier, this issue was not observed with the use of Linux-yocto-dev kernel 6.3.0 version but now, with this kernel as well we observed same.
@khem, I've asked Ross and Steve if the "machine/qemuarm*: don't explicitly set vmalloc " commit can / should be back-ported to mickledore.
@MaxIhlenfeldt @nrpt-m has seen two problems:
More to come tomorrow.
- On dunfell, if you build once, it will succeed for qemuarm for example, then the qemuarm64 build failed and Narpat tried to build a second time for qemuarm, without doing a clean, that build failed as well. I suspect but we haven't yet confirmed that any second build will fail because the new stage added always attempt to re-apply the patches. Narpat to confirm.
More to come tomorrow.
Have confirmed that "any second build is not failing" by building qemuarm on dunfell 2-3 times & it completed without any error.
@kraj I think we are close but, we can't access the network outside the fetch stage. @rwmacleod and I concluded that rather then running npm after patching we could run npm once manually and then generated a patch & save it. Then, we can take @MaxIhlenfeldt patches along with our saved patch and apply them all during the do_patch stage.
We have done that by using npm from the host and unfortunately the patch is 44MB ! We are going to try using the version of npm that shipped with dunfell to see if it generates a smaller patch.
Does it seems like good idea ? Would the 44MB patch be acceptable ? Currently the meta-browser layer is 14MB. chromium-112.0.5615.165-dunfell-rollup-3.12.0-2.58.0-diffstat.txt
@nrpt-m you can enable the network on a particular task this way (for example on compile task):
do_compile[network] = "1"
@ekapllaj yes, one can but that's not a good idea and some OE users need to have the build system and all related source delivered to them so they can do offline builds. That seems kind of outrageous when you're building the chromium browser but those are the requirements that certain organizations have. Given that patching the source code is equivalent to running the 'npm install' command, I think the cost of the rather large 44 MB patch is worthwhile.
That said I do need to become more familiar with npm packaging in Yocto. I acknowledged that the source for the rollup module is < 2 MB and that ideally, we'd figure out how to get npm to fetch from the local source caches rather than accessing the internet. We'd still need to do the fetch during the fetch stage.
I would like to focus on discussing offline building further and exploring other options. The size of the 44MB patch is unreasonable, particularly considering that it will vary with each new release.
@otavio sounds good.
I came to that conclusion about an hour after @nrpt-m and I coughed up the 44 MB patch! ;-)
I'll continue to work on it with @nrpt-m for an hour or two a day unless someone else picks up the ball full-time. Some background is here: https://wiki.yoctoproject.org/wiki/TipsAndTricks/NPM and the various commit logs and some YP documentation may even exist.
Hi all, I still don't have a lot of time for this. I fear it might continue to be like this until W20 (i.e. still no time next week), sorry for that.
Regarding the NPM stuff, do we actually have to run NPM to replace the version of rollup
used? Maybe it's enough to ship https://registry.npmjs.org/rollup/-/rollup-2.58.0.tgz, which is only 1.1 MB, and extract it over the existing dist? I have no idea if that might lead to any dependency problems, but it may be worth a try.
There's also the possibility of adding a new nodejs14 recipe to dunfell as discussed here: https://github.com/OSSystems/meta-browser/pull/706#issuecomment-1518890518
I'm not sure how much work that would be (i.e. how many different architectures need to be tested and what tests to run besides just building the recipe), but it would be the cleanest solution for us, as I expect Chromium to increasingly depend on new rollup features over time.
That would be great of course. But I suppose it would take some time until that's usable, and we probably need a solution for the mean time?
@kraj could you give more details on what kind of testing would be needed for this recipe to be added? Looking at node 14's release notes I don't see many/any breaking changes, in the best case scenario copy-pasting the node recipe from a more recent branch would Just Work.
Hi all, Have updated the nodejs_12.22.12 recipe to nodejs-native_14.18.1 and commented the "0017-Revert-WebUI-Update-Rollup-to-v3.12.0.patch" along with the workaround changes for dunfell in the chromium recipe.
With these above changes, we are able to build the qemux86-64 & rpi4 for dunfell without any error/issue. Now, the build is using nodejs-native_14.18.1, while building the chromium 112.0.5615.165 version. Will build the remaining systems and test.
@rwmacleod Please add, if I have missed anything.
@nrpt-m Good summary. We'll work on some PREFERRED_VERSION changes tomorrow ensure that nodejs_12 is the default for normal uses of meta-oe but that chromium requires and depends on nodejs-native_14.
Hi all, Have updated the nodejs_12.22.12 recipe to nodejs-native_14.18.1 and commented the "0017-Revert-WebUI-Update-Rollup-to-v3.12.0.patch" along with the workaround changes for dunfell in the chromium recipe.
With these above changes, we are able to build the qemux86-64 & rpi4 for dunfell without any error/issue. Now, the build is using nodejs-native_14.18.1, while building the chromium 112.0.5615.165 version. Will build the remaining systems and test.
@rwmacleod Please add, if I have missed anything.
Can you post the nodejs upgrade patch to dunfell mega-oe
@kraj , The patch is not quite ready.
@rwmacleod tells me that he is little rusty about PREFERRED_VERSION settings but, we tried making the following change:
meta-oe/conf/layer.conf +PREFERRED_VERSION_nodejs ?= "12.%" +PREFERRED_VERSION_nodejs-native ?= "14.%"
We were hoping that the native version of nodejs would not matter but, unfortunately, bitbake nodejs(12) fails with the error: ../../deps/v8/src/builtins/array-join.tq:151:8: Torque Error: Parser Error: unexpected token "("
This above error comes from the torque executable running on this tq file. It appears like syntax changed between version 12 and 14. :-(
We are trying not to change nodejs_12 behavior while making nodejs-native_14 available to chromium. Chromium recipe is used in many branches so, without making a branch in meta-browser for dunfell, we don't see how to introduce a dependency on nodejs-native_14 using either a PREFERRED_VERSION or by calling the recipe nodejs-14-native since, that will force all branches to use nodejs-14-native even if they are not using the dunfell branch.
Is there a way to achieve this goal without introducing a branch in meta-browser for dunfell ?
As a test, we have added PREFERRED_VERSION_nodejs-native = "14.%" to conf/local.conf and built bitbake chromium-x11. That picks up nodejs-native_14 as expected and the build completed successfully.
Other solution is that we would required to add PREFERRED_VERSION_nodejs-native = "14.%" to conf/local.conf file while building the chromium for dunfell. More to come tomorrow...
Hi, I actually would prefer the creation of a branch for dunfell. The dunfell distro is going to be around at least until next year (Apr 2024), and I think this issue is comming to bite as at every update. I'm building on Honister and chromium 113 is not compiling at all, so I think it's going to be complicated to try to compile for dunfell for next releases.
I would say that the best compromise would be to kind of freeze the chromium 112 on a separate branch for dunfell (with the appropriate patches) and move on on master branch.
If we choose the dunfell branch, we can branch out on 111 and implement the necessary build fixes. I'm okay with that, but everyone needs to understand that no security updates will be applied.
I'm in the "those who do the work decide" camp. I've been happily reviewing pull requests but haven't worked on updates myself for a while. If the people who have been carrying out the work like @MaxIhlenfeldt, @rwmacleod and @nrpt-m feel like it's no longer worth supporting dunfell (due to either the maintenance burden or lack of client demand), I certainly have no objections.
dunfell users might not be very happy, but it would certainly make updates a lot easier for us.
Hi All,
We had sent the patch to make nodejs_14.18.1 available but not default using DEFAULT_PREFERENCE to make sure that the default version of nodejs remains 12.x. Patch link: https://lore.kernel.org/openembedded-devel/20230511163758.358936-1-narpat.mali@windriver.com/
It's on the way to get it merge into dunfell: https://lore.kernel.org/openembedded-devel/095ad873-2d15-5f66-52fc-5c4e34be3348@gmail.com/
Are you looking to include a note regarding the usage of the dunfell
branch that mentions its restriction?
Are you looking to include a note regarding the usage of the
dunfell
branch that mentions its restriction?
@otavio Have already added the set-up requirements when on dunfell in the PR & commit messages.
Patch to add nodejs 14 has landed in meta-openembedded/dunfell
https://git.openembedded.org/meta-openembedded/commit/?h=dunfell&id=116bfe8d5e5851e7fc5424f40da8691a19c5b5ee
Looks like we're finally good to go. @MaxIhlenfeldt @rafluan @kraj please merge if you agree.
@nrpt-m Can you address these comments and upload again
Thanks a lot for finishing this! Almost LGTM, just some polishing comments.
Can you please remove the mentions of
0017-Revert-WebUI-Update-Rollup-to-v3.12.0.patch
and0019-Backport-Fix-deps-for-feed_response.proto.patch
from the PR description? Not sure if it makes sense to also squash the commits, so that the new single commit message matches the description?
Have removed the mentions of 0017-Revert-WebUI-Update-Rollup-to-v3.12.0.patch
and 0019-Backport-Fix-deps-for-feed_response.proto.patch
from the PR description but, kept as it is in commit message.
@nrpt-m Can you address these comments and upload again
Done. Shall I move "fix-libc-version-include.patch" patch into musl/ as @MaxIhlenfeldt suggested ?
@nrpt-m Can you address these comments and upload again
Done. Shall I move "fix-libc-version-include.patch" patch into musl/ as @MaxIhlenfeldt suggested ?
yes
@MaxIhlenfeldt , @kraj Requested changes has been done and let me know if any further modification needed.
Release notes: https://chromereleases.googleblog.com/2023/04/stable-channel-update-for-desktop.html (112.0.5615.49) https://chromereleases.googleblog.com/2023/04/stable-channel-update-for-desktop_14.html (112.0.5615.121) https://chromereleases.googleblog.com/2023/04/stable-channel-update-for-desktop_18.html (112.0.5615.165)
Build and patch changes:
Renamed & added 0017-Revert-Fix-std-span-autodetection-7805.patch to enable building with clang 12.
Renamed & added 0018-Only-default-operator-on-declaration.patch to enable building with clang12, which doesn't implement P2085R0 and therefore requires all defaulted operator== implementations to be defined on the first declaration.
Moved fix-libc-version-include.patch to musl.
Rebase remaining patches.
License changes:
Added licenses:
third_party/android_deps/libs/com_google_android_play_core_common/LICENSE
third_party/android_deps/libs/com_google_android_play_feature_delivery/LICENSE
https://chromium-review.googlesource.com/c/chromium/src/+/4226920
third_party/android_rust_toolchain/toolchain/share/doc/rust/LICENSE-APACHE
https://chromium-review.googlesource.com/c/chromium/src/+/4275302
third_party/devtools-frontend/src/front_end/third_party/lit/LICENSE
https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/4060452
third_party/devtools-frontend/src/front_end/third_party/puppeteer/LICENSE
https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/4275342
Removed licenses:
third_party/android_deps/libs/com_google_android_play_core/LICENSE
https://chromium-review.googlesource.com/c/chromium/src/+/4233789
third_party/android_deps/libs/org_jetbrains_annotations/LICENSE
third_party/android_deps/libs/org_jetbrains_kotlin_kotlin_stdlib/LICENSE
third_party/android_deps/libs/org_jetbrains_kotlin_kotlin_stdlib_common/LICENSE
https://chromium-review.googlesource.com/c/chromium/src/+/4226232
Updated licenses:
third_party/android_deps/libs/com_google_android_gms_play_services_basement/LICENSE
third_party/android_deps/libs/com_google_android_gms_play_services_tasks/LICENSE
https://chromium-review.googlesource.com/c/chromium/src/+/4226920
third_party/fontconfig/src/COPYING
https://gitlab.freedesktop.org/fontconfig/fontconfig/-/commit/3ae08a8, https://gitlab.freedesktop.org/fontconfig/fontconfig/-/commit/74f0595, https://gitlab.freedesktop.org/fontconfig/fontconfig/-/commit/23cede3, https://chromium-review.googlesource.com/c/chromium/src/+/4251898
third_party/hyphenation-patterns/LICENSE
https://chromium-review.googlesource.com/c/chromium/src/+/4189277
Test-built:
chromium-ozone-wayland:
chromium-x11
** Please note that Chromium requires below set-up when on dunfell branch.
Signed-off-by: Narpat Mali narpat.mali@windriver.com