pbnjson fails to build

[rossburton]

With v2.24.0 of meta-webosose, pbnjson fails to build:

FAILED: src/pbnjson_c/selectors/CMakeFiles/json_selectors.dir/selectors_grammar_y.c.o
TMPDIR/work/qemux86_64-webos-linux/libpbnjson/2.15.0-13-r15/recipe-sysroot-native/usr/bin/x86_64-webos-linux/x86_64-webos-linux-gcc --sysroot=TMPDIR/work/qemux86_64-webos-linux/libpbnjson/2.15.0-13-r15/recipe-sysroot -DWEBOS_COMPONENT_VERSION=\"2.15.0\" -D_GNU_SOURCE -ITMPDIR/work/qemux86_64-webos-linux/libpbnjson/2.15.0-13-r15/git/include/public -ITMPDIR/work/qemux86_64-webos-linux/libpbnjson/2.15.0-13-r15/recipe-sysroot/usr/include/glib-2.0 -ITMPDIR/work/qemux86_64-webos-linux/libpbnjson/2.15.0-13-r15/recipe-sysroot/usr/lib/glib-2.0/include -ITMPDIR/work/qemux86_64-webos-linux/libpbnjson/2.15.0-13-r15/recipe-sysroot/usr/include/yajl -ITMPDIR/work/qemux86_64-webos-linux/libpbnjson/2.15.0-13-r15/git/include/public/pbnjson -ITMPDIR/work/qemux86_64-webos-linux/libpbnjson/2.15.0-13-r15/git/include/public/pbnjson/c -ITMPDIR/work/qemux86_64-webos-linux/libpbnjson/2.15.0-13-r15/build/src/pbnjson_c/selectors -ITMPDIR/work/qemux86_64-webos-linux/libpbnjson/2.15.0-13-r15/build/src/pbnjson_c/selectors/.. -ITMPDIR/work/qemux86_64-webos-linux/libpbnjson/2.15.0-13-r15/git/src/pbnjson_c/selectors -m64 -march=core2 -mtune=core2 -msse3 -mfpmath=sse -fstack-protector-strong  -O2 -D_FORTIFY_SOURCE=2 -Wformat -Wformat-security -Werror=format-security -Werror=return-type  --sysroot=TMPDIR/work/qemux86_64-webos-linux/libpbnjson/2.15.0-13-r15/recipe-sysroot  -O2 -pipe -g -feliminate-unused-debug-types -fmacro-prefix-map=TMPDIR/work/qemux86_64-webos-linux/libpbnjson/2.15.0-13-r15=/usr/src/debug/libpbnjson/2.15.0-13-r15                      -fdebug-prefix-map=TMPDIR/work/qemux86_64-webos-linux/libpbnjson/2.15.0-13-r15=/usr/src/debug/libpbnjson/2.15.0-13-r15                      -fdebug-prefix-map=TMPDIR/work/qemux86_64-webos-linux/libpbnjson/2.15.0-13-r15/recipe-sysroot=                      -fdebug-prefix-map=TMPDIR/work/qemux86_64-webos-linux/libpbnjson/2.15.0-13-r15/recipe-sysroot-native=   -Wall -D__STRICT_ANSI__ -pthread -fmessage-length=0 -Wall -Wl,--no-undefined -std=c11 -fPIC -fPIC -DNDEBUG -DNDEBUG -g -Wno-unused-variable -MD -MT src/pbnjson_c/selectors/CMakeFiles/json_selectors.dir/selectors_grammar_y.c.o -MF src/pbnjson_c/selectors/CMakeFiles/json_selectors.dir/selectors_grammar_y.c.o.d -o src/pbnjson_c/selectors/CMakeFiles/json_selectors.dir/selectors_grammar_y.c.o -c TMPDIR/work/qemux86_64-webos-linux/libpbnjson/2.15.0-13-r15/build/src/pbnjson_c/selectors/selectors_grammar_y.c
selectors_grammar_y.y: In function 'yy_destructor':
selectors_grammar_y.y:70:25: error: 'YYMINORTYPE' has no member named 'yy'; did you mean 'yy0'?
selectors_grammar_y.y:382:20: error: 'YYMINORTYPE' has no member named 'yy'; did you mean 'yy0'?
selectors_grammar_y.y:349:24: error: 'YYMINORTYPE' has no member named 'yy'; did you mean 'yy0'?
selectors_grammar_y.y: In function 'yy_reduce':
selectors_grammar_y.y:66:41: error: 'YYMINORTYPE' has no member named 'yy'; did you mean 'yy0'?
selectors_grammar_y.y:73:17: error: 'YYMINORTYPE' has no member named 'yy'; did you mean 'yy0'?
[ ... ]

I've previously seen lemon segfault, but that didn't happen when I rebuild pbnjson with an empty tmpdir.

[hieunv0828]

Hi @rossburton ,

Could you please share your build system information and the build configuration? I have tried to build many times (from the scratch), but I have not encountered this problem.

Build System:

1) Description:    Ubuntu 22.04.2 LTS
    Release:        22.04
2) Description:    Ubuntu 18.04.6 LTS // with GCC-9
    Release:        18.04

Build Configuration:                                                                                                                               
BB_VERSION           = "2.0.0"                                                                                                                     
BUILD_SYS            = "x86_64-linux"                                                                                                              
NATIVELSBSTRING      = "universal"                                                                                                                 
TARGET_SYS           = "x86_64-webos-linux"                                                                                                        
MACHINE              = "qemux86-64"
DISTRO               = "webos"
DISTRO_VERSION       = "2.24.0.g"
TUNE_FEATURES        = "m64 core2"
TARGET_FPU           = ""
WEBOS_DISTRO_RELEASE_CODENAME = "webos-master"
WEBOS_DISTRO_BUILD_ID = "unofficial"
WEBOS_DISTRO_TOPDIR_REVISION = "3c0fd6d7380e5c459ce362244467a2b4ed7e8894"
WEBOS_DISTRO_TOPDIR_DESCRIBE = "builds/master/659"
DATETIME             = "20231219143154"
meta-webos
meta-webos-backports-4.3
meta-webos-backports-4.2
meta-webos-backports-4.1 = "master:a9afa03412dfbbacd01db4e7a276f83c0099c834"
meta-qt6             = "6.5.2:74d7d77e23b144cbb801cf6af7ec78b9673158f4"
meta-clang           = "kirkstone:1602c0ad2a0dc011a3fac94b178a5b9935d78b80"
meta-filesystems
meta-python
meta-networking
meta-multimedia
meta-oe              = "kirkstone:402affcc073db39f782c1ebfd718edd5f11eed4c"
meta                 = "kirkstone:eea685e1caafd8e8121006d3f8b5d0b8a4f2a933"

[rossburton]

I did a number of builds on different machines and have concluded that this is an aarch64 host issue:

• Ubuntu 22.04 on x86 works
• Ubuntu 20.04 on x86 works
• Ubuntu 22.04 on aarch64 fails
• Ubuntu 20.04 on aarch64 fails

I'm guessing the problem is the lemon tool as the generated selectors_grammer.y.c is different, despite the target being identical. Attached are the two files.

grammers.zip

Notable differences include the YYMINORTYPE union having less fields, and assignments like this which are clearly broken:

yygotominor.yy-84 = jnumber_create_i64(_int);

[rossburton]

Also as we need to build on the AB without build-webos, I've been using Kas locally to orchestrate the build in a way that I can easily transform to autobuilder configuration when it works.

This is the kas file I've been using: webos.yml.txt.

[rossburton]

I noticed that lemon in meta-oe is very old. I upgraded it and now the errors change to:

selectors_grammar_y.y: In function 'yy_syntax_error':
selectors_grammar_y.y:47:17: error: 'raw_buffer' has no member named 'yy0'
selectors_grammar_y.y:56:37: error: 'raw_buffer' has no member named 'yy0'

[rossburton]

Lemon upgrade here https://lore.kernel.org/openembedded-devel/20231220155334.1055853-1-ross.burton@arm.com/

[hieunv0828]

@rossburton, thank you for the information.

We have never used any aarch64 machines, and we currently don't need to verify host aarch64 build yet. We might have problems with other webOS packages when build on an aarch64 machine, not just libpbnjson. So could you please verify builds only on X86 machines as you did before.

Please let me know if you have any issues that block the integration of webOS into autobuilder.

[rossburton]

It looks like this was a genuine bug in lemon that has been fixed in the last decade. I suggest you grab that lemon upgrade: with that the generated files are identical on both platforms. The yy0 error is most likely fallout from changes to lemon and easily fixed.

[hieunv0828]

Hi @rossburton,

I tried to fix the yy0 error and now libpbnjson can be built with lemon-3.44.2.

https://lore.kernel.org/openembedded-devel/20231220155334.1055853-1-ross.burton@arm.com/

After your patches are merged into meta-oe, I will update the libpbnjson recipe in webosose layer.

selectors_grammar_y.y.diff:

diff --git a/src/pbnjson_c/selectors/selectors_grammar_y.y b/src/pbnjson_c/selectors/selectors_grammar_y.y
index 006c16f..6b3c642 100644
--- a/src/pbnjson_c/selectors/selectors_grammar_y.y
+++ b/src/pbnjson_c/selectors/selectors_grammar_y.y
@@ -44,7 +44,7 @@
 }

 %syntax_error {
-    if (!yyminor.yy0.m_str)
+    if (!yyminor.m_str)
     {
         jerror_set(context->error, JERROR_TYPE_SYNTAX,
                    "Unexpected end of the query string");
@@ -53,7 +53,7 @@
     {
         jerror_set_formatted(context->error, JERROR_TYPE_SYNTAX,
                              "Unexpected token '%s' in the query string",
-                             yyminor.yy0.m_str);
+                             yyminor.m_str);
     }
 }

[hieunv0828]

@rossburton, We have backported lemon-3.44.2 from your commit in meta-oe, and fixed the libpbnjson (https://github.com/webosose/meta-webosose/commit/988312d065961d1e499ad9311e68bc9f4ca566a2, https://github.com/webosose/meta-webosose/commit/d3582b347e328f7847108e08cd8649ab051a5a69). I hope it can be build on an aarch64 machine.

[jaeyoonjung]

@rossburton < https://github.com/webosose/meta-webosose/commit/988312d065961d1e499ad9311e68bc9f4ca566a2 https://github.com/webosose/meta-webosose/commit/d3582b347e328f7847108e08cd8649ab051a5a69 This should fix this issue. Please verify it and close this issue.