[Cross build ARM] 20.1 config ready to be use?

[nekoprog]

Just wondering if it's okay to build using 20.1 config, because I encounter some error when running make base SETTINGS=20.1 with FreeBSD 12.0. Looks like the base.txz not found, but the script was looking at wrong directory where it supposed to look at STAGEDIR/usr/src, but it went to host /usr/src.

Will update the error with log later, I just close my VM.

[fichtner]

src.git master is 12.1 and I haven't seen that issue so yes you can use 20.1, but please use our src.git

[nekoprog]

this is the error that i found

it supposed to be /usr/obj/usr/src/arm.armv6/release right?

[fichtner]

Hmmm, maybe it's different now for cross vs. native builds?

[nekoprog]

I can confirm that the error occurs only when using FreeBSD 12.1 BETA release as host, error for both 19.7 and 20.1

No such error when using FreeBSD 11.2 as host. Testing for 12.1-RC1 today.

[nekoprog]

Same error on FreeBSD11.2 but with tools.git/master. Works fine from tag 19.7.4 and below.

For some unknown reason, a variable storing this location /usr/obj/usr/src/arm.armv6/release returns to /usr/src/release/base.txz without /usr/obj.

base.txz is generated ls /usr/obj/usr/src/arm.armv6/release. I guess it's just a minor error that can be fixed soon.

[fichtner]

It might be a local issue with your src.git or /usr/obj. I cannot reproduce on any version at the moment.

[rene-bayer]

Same here on a fresh installed 20.1 ... 19.7 worked fine on bsd 11.2

[fichtner]

This is also a cross build, isn't it? If you can reproduce you can fix. I'm not at the point to look at this yet.

[nekoprog]

Yes, cross build issue. Maybe there are some changes in chroot, buildworld, buildkernel, etc. But don't have spare time to debug yet.

From what I found before is that ${BASE_OBJ} doesn't include directory set on STAGEDIRPREFIX

[rene-bayer]

Thanks for the hint, i think i fixed it ... first compile looks good so far, but need some time to test, if it actually works (boot and so on)

Will create a pr when finished

[fichtner]

very nice, thanks in advance! ❤️

[nekoprog]

Nice, looking forward for a fix.

[rene-bayer]

base and kernel looks good so far, but didn't tested with 19.7 settings a.t.m ...

struggling around with xtools and packages now ... i think i'll try to add native cross compiler support, without all that qemu stuff... as far i found out, this should be possible with bsd12 :-) but still facing strange toolchain errors :-(

[rene-bayer]

Little update: compiling base, kernel and xtools works ... trying to fix packages now

Edit: when everything is done, i should be able to run OPNsense on my Netgate SG3100 appliance .. this has been my initial goal ^^

[rene-bayer]

i think i am very close now.

My current problem is, that the pkg-list is not generated in my stagedir/.pkg/All

Anyone an idea?

Edit: or in which step the pkg list should be generated?

[nekoprog]

You need to trace PACKAGESDIR from build/common.sh. There are many steps involving *_packages(). If error on make packages, you can start from extract_packages() and bundle_packages()

[rene-bayer]

thanks for the hint again.

in fact there was a problem in the configdir/make.conf ... compiling ports now :-)

[rene-bayer]

Ran straight into the next bug :-(

The compiling process hangs at "checking whether printf survives out-of-memory conditions..."

Seems to be this bug: https://lists.freebsd.org/pipermail/freebsd-ports/2017-December/112090.html https://lists.gnu.org/archive/html/qemu-devel/2015-02/msg05584.html

Edit: found out, that the build continues when i send a kill -9 to the stuck process

[nekoprog]

I need more time to investigate what happen from this commit. It seems like somewhere in SRCDIR/Makefile, buildworld is messing up with cross build env.

These lines 340 and 345 keeps bothering me about not bootstrapping. Maybe this is the culprit, maybe not. I'll try changing these values and build something.

Update: I don't have that much spare time to dissect each codes. Make some dirty hacks to some of these files base.sh kernel.sh arm.sh. Works OK for now. Waiting for make packages to finish and then make arm image to see it boots or not.

[rene-bayer]

Did your package build finished?

On armv7 it kicks me out at openssl with the message, that my compiler doesnt support 128bit integer and on arm64 (rpi3) it cant compile lang/rust :-(

Building on 12.1 RC2

[nekoprog]

Nah, stuck at make xtools. It's says nxb-bin not found and it just generating xtools.txz with zero byte. Will look at it later.

When running make packages, got error about fatal errors encountered -- cannot continue cp: /usr/obj/usr/tools/config/20.1/OpenSSL:armv7/.pkg/All/*: no such file or directory.

Update: Fixed on https://github.com/opnsense/tools/commit/80e28f5e8dc53e3b709b07cad7984bd707c7c053

[rene-bayer]

Look at my fork, already fixed it there

[nekoprog]

build/ports.sh - Line 119 produces this error:

make: "/etc/make.conf" line 0: 1 open conditional
make: Fatal errors encountered -- cannot continue
[: make:: unexpected operator

I don't know why it says line 0 has 1 open conditional. So I stuck at make packages. We are almost there @DarkSunOne.

Update: Fixed on https://github.com/opnsense/tools/commit/80e28f5e8dc53e3b709b07cad7984bd707c7c053

[rene-bayer]

Take the Makefile.inc1 from my tools fork and put it in your /usr/src dir :-)

[rene-bayer]

Next little process :-) packages succeeded, struggling around with core now, saying that suricata-devel was not found (commented out cause of this

Edit1: Oh my god, it seems that it just passes rust .. fingers crossed :-) Edit2: ... nevermind ... still no luck with rust

[fichtner]

I removed suricata/rust from arm in tools.git and core.git. Please update your branches.

[rene-bayer]

Is a "clean-packages" necessary? I did so cause it still complained that suricata-devel is missing (with updated branches)

[fichtner]

This should do the trick after repo updates:

# make hotfix

On 29. Oct 2019, at 12:26, René Losert notifications@github.com wrote:

 Is a "clean-packages" necessary? I did so cause it still complained that suricata-devel is missing (with updated branches)

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

[rene-bayer]

Still complains missing suricata-devel during make core :-(

[fichtner]

Can’t be, did you update core.git ?

On 29. Oct 2019, at 16:35, René Losert notifications@github.com wrote:

 Still complains missing suricata-devel during make core :-(

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

[rene-bayer]

I manually merged the changes to my files, shouldn't be a problem?

[fichtner]

Well, unless architectures are i386 or amd64 suricata-devel shouldn't be a core dependency anymore:

https://github.com/opnsense/core/commit/f098b3a9ba

[rene-bayer]

Maybe a problem with target detection? Currently compiling for armv7 on an amd64 host Same problem when compiling for arm64

[fichtner]

I don't understand. core.git depends on suricata-devel, but if you not build i386 or amd64 it won't use suricata-devel anymore as per the referenced source commit. Make sure the core.git master branch is in sync and checked in.

[nekoprog]

I don't have any problem with suricata-devel and rust. How did you run your build @DarkSunOne? Did you include DEVICE argument?

[fichtner]

Created a portable fix for the xtools on version 12: a60abedda

[rene-bayer]

This file has been added to automatically load the installed extension: /usr/local/etc/php/ext-20-gettext.ini Could not find package: php72-google-api-php-client *** Error code 1

[fichtner]

@DarkSunOne sorry, moving target. your ports now needs to have php72-google-api-php-client package for core to work

# make ports-again SOMEARGS...

It will flush plugins and core and build the missing packages like the google api one

[rene-bayer]

Perfect, got all sets ready to go.

Currently stuck at make arm, complaining that package os-dyndns is not found, even if its there.

If you dont have time, i will look at it in 2-3 days

[nekoprog]

Ran straight into the next bug :-(

The compiling process hangs at "checking whether printf survives out-of-memory conditions..."

Seems to be this bug: https://lists.freebsd.org/pipermail/freebsd-ports/2017-December/112090.html https://lists.gnu.org/archive/html/qemu-devel/2015-02/msg05584.html

Edit: found out, that the build continues when i send a kill -9 to the stuck process

@fichtner How to implement this globally? I tried pasting it inside config/20.1/make.conf, but qemu-arm-static still hangs on ports/devel/m4 ports/devel/bison ports/devel/libunistring ports/editors/nano (or maybe something else that cause it, because makefile already implemented it). Need to manually kill /usr/local/bin/qemu-arm-static ./conftest so that build continues.

.ifdef QEMU_EMULATING
# XXX bug 224740: configure hangs: GSlice: failed to allocate 496 bytes (alignment: 512)
CONFIGURE_ENV+= gl_cv_func_printf_enomem=no
.endif

@DarkSunOne Last time I got an error about os-dyndns not found, I just ran make packages again.

[dgktkr]

opnsense/src master is HardenedBSD on top of FreeBSD 12.1? I've looked at the online display of the file sys/conf/options and there seem to be no PAX* entries present. As a confirmation, I cloned the whole OPNSense src.git tree manually to another computer and the PAX* kernel options are also absent from sys/conf/options. Am I doing something wrong?

Interestingly enough, in GitHub opnsense/src/sys/conf/options for branch stable/19.7 there are lots of PAX_* kernel options.

[nekoprog]

opnsense/src master is HardenedBSD on top of FreeBSD 12.1? I've looked at the online display of the file sys/conf/options and there seem to be no PAX* entries present. As a confirmation, I cloned the whole OPNSense src.git tree manually to another computer and the PAX* kernel options are also absent from sys/conf/options. Am I doing something wrong?

Interestingly enough, in GitHub opnsense/src/sys/conf/options for branch stable/19.7 there are lots of PAX_* kernel options.

Maybe they still patching for the upcoming 12.1 stable release which is tomorrow.

[fichtner]

Src.git master is a work in progress. Currently we have FreeBSD releng/12.1 there. HardenedBSD is not yet imported due to merge conflicts between the two.

On 3. Nov 2019, at 05:40, Neko Prog notifications@github.com wrote:

 opnsense/src master is HardenedBSD on top of FreeBSD 12.1? I've looked at the online display of the file sys/conf/options and there seem to be no PAX* entries present. As a confirmation, I cloned the whole OPNSense src.git tree manually to another computer and the PAX* kernel options are also absent from sys/conf/options. Am I doing something wrong?

Interestingly enough, in GitHub opnsense/src/sys/conf/options for branch stable/19.7 there are lots of PAX_* kernel options.

Maybe they still patching for the upcoming 12.1 stable release which is tomorrow.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[nekoprog]

I will close this issue for now since all sets have been build successfully on my fork. Bugs regarding qemu has been submitted on bugzilla.

Hoping to see openssl111 on arm next. Since current compiler doesn't support 128-bit integer when compiling. Also miniupnpd is unable to be compiled in OPNsense 20.1, hope to see that fixed too.

[fichtner]

We could prefer LibreSSL for ARM. And I think I know the issue for miniupnpd... https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=233963 ... right now we have a patch because on FreeBSD 11 the build is broken... ;)

[nekoprog]

Nice, can't wait for the HBSD patch. Will test tools.git again after that.

[rene-bayer]

Openssl works, even with arm and 20.1 settings, you just have to disable the 128bit compiler support in the makefile of openssl.

For miniupnpd there you just have to disable a port check feature which was removed from FreeBSD 12, i dont remember the exact settings, will post the links tomorrow.

[dgktkr]

For miniupnpd there you just have to disable a port check feature which was removed from FreeBSD 12, i dont remember the exact settings, will post the links tomorrow.

DarkSunOne, how exactly did you deal with miniupnp? After 8 hours of working on the packages set, my build errors out because

Could not find package: miniupnp

[nekoprog]

You need to add ignore arm,arm64 to plugins.conf/upnp and ports.conf/miniupnpd

[rene-bayer]

@dgktkr or you remove CHECK_PORTINUSE in the make.conf