Added snort3 & libdaq recipes

[kraj]

there is a build failure as well - https://errors.yoctoproject.org/Errors/Details/752967/

[kraj]

also make sure its only enabled for architectures supporting luajit otherwise we see errors like this

12:09:02 luajit was skipped: incompatible with host riscv64-yoe-linux (not in COMPATIBLE_HOST)
12:09:10 ERROR: Nothing RPROVIDES 'snort3-dev' (but /mnt/jenkins/workspace/yocto-world-glibc/sources/meta-openembedded/meta-networking/recipes-connectivity/snort/snort3_git.bb RDEPENDS on or otherwise requires it)
12:09:10 No eligible RPROVIDERs exist for 'snort3-dev'
12:09:10 NOTE: Runtime target 'snort3-dev' is unbuildable, removing...
12:09:10 Missing or unbuildable dependency chain was: ['snort3-dev']
12:09:15 ERROR: Nothing RPROVIDES 'snort3' (but /mnt/jenkins/workspace/yocto-world-glibc/sources/meta-openembedded/meta-networking/recipes-connectivity/snort/snort3_git.bb RDEPENDS on or otherwise requires it)
12:09:15 No eligible RPROVIDERs exist for 'snort3'
12:09:15 NOTE: Runtime target 'snort3' is unbuildable, removing...
12:09:15 Missing or unbuildable dependency chain was: ['snort3']

[RNGesus-exe]

@kraj Can you tell me how to reproduce the above errors on my end. I ended up isolating the bash dependent script into a seperate package but I want to get the errors on my end so I can fix luajit + yocto build error

[kraj]

@kraj Can you tell me how to reproduce the above errors on my end. I ended up isolating the bash dependent script into a seperate package but I want to get the errors on my end so I can fix luajit + yocto build error

I am building with clang/libc++, one way to reproduce it would be to use yoe distro https://github.com/yoedistro/yoe-distro

. ./envsetup.sh qemuriscv64
bitbake snort3

[RNGesus-exe]

I added the following code block to my snort3 recipe, is this what you want? I read the luajit recipe and apparently, it is not supported for the following architectures yet. I reproduced the error using the approach you gave above as well.

# mips64/ppc/ppc64/riscv64/riscv32 is not supported
COMPATIBLE_HOST:mipsarchn32 = "null"
COMPATIBLE_HOST:mipsarchn64 = "null"
COMPATIBLE_HOST:powerpc = "null"
COMPATIBLE_HOST:powerpc64 = "null"
COMPATIBLE_HOST:powerpc64le = "null"
COMPATIBLE_HOST:riscv64 = "null"
COMPATIBLE_HOST:riscv32 = "null"

[kraj]

Yes that would fix the compatiblity errror I am still worried about the build failures seen with clang and libc++

[RNGesus-exe]

Yup, that build error is still occurring for qemuarm, I am resolving it for now

[RNGesus-exe]

Update: I fixed the build error for qemuarm machine, by passing "-lm" flag to the libdaq recipe

LDFLAG += "-lm"

Now, libdaq is building but when building snort3, I get a new error something as below

/yoe/build/tmp/work/cortexa15t2hf-neon-yoe-linux-gnueabi/snort3/git/git/src/helpers/process.cc:191:56: error: use of undeclared identifier 'unw_strerror'; did you mean 'strerror'?
|   191 |         ssp.printf("unw_getcontext failed: %s (%d)\n", unw_strerror(ret), ret);
|       |                                                        ^~~~~~~~~~~~
|       |                                                        strerror

I added libunwind in DEPENDS and passed -lunwind as LDFLAG but the error still occurring so trying to resolve that

[RNGesus-exe]

@kraj I have created a patch (which, I'll push once the following issue gets resolved) for the llibdaq build error as show below

| arm-yoe-linux-gnueabi-ld.lld: error: undefined symbol: ceilf
| >>> referenced by math.h:632 (/usr/include/c++/v1/math.h:632)
| >>>               libdaq_static_fst_la-daq_fst.o:(std::__1::pair<std::__1::__hash_iterator<std::__1::__hash_node<std::__1::__hash_value_type<FstKey, FstNode*>, void*>*>, bool> std::__1::__hash_table<std::__1::__hash_value_type<FstKey, FstNode*>, std::__1::__unordered_map_hasher<FstKey, std::__1::__hash_value_type<FstKey, FstNode*>, FstKeyHash, std::__1::equal_to<FstKey>, true>, std::__1::__unordered_map_equal<FstKey, std::__1::__hash_value_type<FstKey, FstNode*>, std::__1::equal_to<FstKey>, FstKeyHash, true>, std::__1::allocator<std::__1::__hash_value_type<FstKey, FstNode*>>>::__emplace_unique_key_args<FstKey, std::__1::pair<FstKey const, FstNode*>>(FstKey const&, std::__1::pair<FstKey const, FstNode*>&&)) in archive ../modules/fst/.libs/libdaq_static_fst.a

libdaq is working now but snort3 produces a build error as follows

/yoe/build/tmp/work/cortexa15t2hf-neon-yoe-linux-gnueabi/snort3/git/git/src/helpers/process.cc:191:56: error: use of undeclared identifier 'unw_strerror'; did you mean 'strerror'?
|   191 |         ssp.printf("unw_getcontext failed: %s (%d)\n", unw_strerror(ret), ret);
|       |                                                        ^~~~~~~~~~~~
|       |                                                        strerror

This is the exact line number where the error is produced during snort3 compilation process.cc line 191 error The library libunwind.h on github has the function unw_strerror but when I checked the meta-clang layer configuration, I noticed this meta_clang layer.conf

So, to sum it up meta_clang provides lib_cxx and lib_cxx provides libunwind (which does not have the unw_sterror). My goal is not to add clang support to snort3 so, Can you tell me how to add incompatibility for snort3 using clang.

[kraj]

This error means a dependency on libunwind is missing.

[RNGesus-exe]

This error means a dependency on libunwind is missing.

I added it already, but it doesn't work. As meta-clang provides it's own libunwind

[kraj]

Can you set TC_CXX_RUNTIME = "gnu" and see if that helps ?

[RNGesus-exe]

Can you set TC_CXX_RUNTIME = "gnu" and see if that helps ?

same error

[kraj]

Can you set TC_CXX_RUNTIME = "gnu" and see if that helps ?

same error

Add it in meta-clang/conf/nonclangable.conf like below

TC_CXX_RUNTIME:pn-snort3:toolchain-clang = "gnu"

[RNGesus-exe]

Can you set TC_CXX_RUNTIME = "gnu" and see if that helps ?

same error

Add it in meta-clang/conf/nonclangable.conf like below

TC_CXX_RUNTIME:pn-snort3:toolchain-clang = "gnu"

nope, still the same error Yocto Build Error

[RNGesus-exe]

I checked the environment bitbake -e libunwind, it is still being provided by libcxx

FILE="/home/rngesusexe/Programming/yocto/yoe/sources/meta-clang/recipes-devtools/clang/libcxx_git.bb"

[kraj]

OK, can you post your updated patches to the PR, I want to take a look.

[kraj]

Check for TC_CXX_RUNTIME what is it set to

[RNGesus-exe]

Check for TC_CXX_RUNTIME what is it set to

I have pushed the new changes, but here the variable is set as following TC_CXX_RUNTIME="llvm"

[kraj]

Check for TC_CXX_RUNTIME what is it set to

I have pushed the new changes, but here the variable is set as following TC_CXX_RUNTIME="llvm"

Even when you set to gnu ? If so then that’s the issue

[RNGesus-exe]

Check for TC_CXX_RUNTIME what is it set to

I have pushed the new changes, but here the variable is set as following TC_CXX_RUNTIME="llvm"

Even when you set to gnu ? If so then that’s the issue

Yup I added in meta-clang/conf/nonclangable.conf TC_CXX_RUNTIME:pn-snort3:toolchain-clang = "gnu"

[RNGesus-exe]

@kraj is there any way to make libunwind from poky/meta/recipes-support/libunwind/libunwind_1.6.2.bb be used instead of the meta-clang one.

[kraj]

Please test https://github.com/openembedded/meta-openembedded/commit/8db172c5fe200053447c2df57a3d760a7efa451c

this should fix the problem.

[kraj]

Addition QA warnings

WARNING: snort3-3+git-r0 do_package_qa: QA Issue: File /usr/lib/pkgconfig/snort.pc in package snort3-dev contains reference to TMPDIR [buildpaths]
WARNING: snort3-3+git-r0 do_package_qa: QA Issue: File /usr/src/debug/snort3/3+git/src/js_norm/pdf_tokenizer.cc in package snort3-src contains reference to TMPDIR
File /usr/src/debug/snort3/3+git/src/js_norm/js_tokenizer.cc in package snort3-src contains reference to TMPDIR [buildpaths]

For once I have addressed them via

https://github.com/openembedded/meta-openembedded/commit/973647848049abda6f2adbee0189720109863731

However, I expect submitters to be able to handle these in general.

[RNGesus-exe]

Please test 8db172c

this should fix the problem.

Tested, it is working

[RNGesus-exe]

Addition QA warnings

WARNING: snort3-3+git-r0 do_package_qa: QA Issue: File /usr/lib/pkgconfig/snort.pc in package snort3-dev contains reference to TMPDIR [buildpaths]
WARNING: snort3-3+git-r0 do_package_qa: QA Issue: File /usr/src/debug/snort3/3+git/src/js_norm/pdf_tokenizer.cc in package snort3-src contains reference to TMPDIR
File /usr/src/debug/snort3/3+git/src/js_norm/js_tokenizer.cc in package snort3-src contains reference to TMPDIR [buildpaths]

For once I have addressed them via

9736478

However, I expect submitters to be able to handle these in general.

I didn't have these QA warnings at my end as the build was failing or else I would have resolved them, nonetheless thankyou for resolving them from your side.

[kraj]

applied to master branch as of 7132d7b571

[shr-project]

In multilib builds I'm also seeing:

ERROR: lib32-snort3-3+git-r0 do_populate_sysroot: QA Issue: snort.pc failed sanity test (tmpdir) in path lib32-snort3/3+git/sysroot-destdir/usr/lib/pkgconfig [pkgconfig]

And the ${SRCPV} shouldn't be in PV, see: https://git.openembedded.org/meta-openembedded/commit/?id=21f956598ddb1bb08855fbc9a3293acf957154d1

[shr-project]

I'll send fixes for both issues to ML shortly.

[RNGesus-exe]

@shr-project How can I reproduce this error

In multilib builds I'm also seeing:

ERROR: lib32-snort3-3+git-r0 do_populate_sysroot: QA Issue: snort.pc failed sanity test (tmpdir) in path lib32-snort3/3+git/sysroot-destdir/usr/lib/pkgconfig [pkgconfig]

moreover can you kindly give some info on how to resolve this issue.

[shr-project]

See the fixes I've sent to ML and were already merged (see https://git.openembedded.org/meta-openembedded/log/?qt=grep&q=snort3)