Getting source tree for an image

[mburnicki]

Hi, I'm not very familiar with FreeBSD but have successfully ported a driver package which runs fine on a PC with FreeBSD 8 / 9 / 10 / 11 / 12 on i386 / amd64 architecture.

Now I want to see if it also builds and works fine on ARM, so I downloaded and installed http://download.raspbsd.org/FreeBSD-armv6-12.0-RPI2-310476.img.gz

This works properly on an RPi3, but building the kernel driver on the RPi3 fails since the source tree is missing.

When installing FreeBSD on a PC I can mark the source tree to be installed normally during installation. Can someone please give me a hint how to install the proper source tree on my RPi3?

Thanks!

[arrowd]

If you have a running ARM system, you can just checkout source tree via git or SVN. Clone github.com/freebsd/freebsd into /usr/src, for instance.

[mburnicki]

Thanks, that basically worked. The kernel module now compiles fine, without errors or warnings, but I still get an error when the kernel module is linked: ld -m aarch64elf -d -warn-common -r -d -o mbgclock.kld mbgclock_main.o pcpsdrvr.o identdec.o rsrc_bsd.o unknown emulation: aarch64elf

Anyone has a hint what can be the problem? I haven't specified any cross/target environment, it's just a native build on the Pi.

[rozmansi]

I'm not very familiar with FreeBSD either, but isn't FreeBSD-armv6-12.0-RPI2-310476.img.gz actually armv6 architecture (32-bit ARM), and you are trying to compile aarch64 (64-bit ARM)?

[brd]

It is, yes.

[mburnicki]

Hm, AFAICS that does not seem to be the case.

"uname -a" run on the RPi3 ends with "arm64", so this seems to be indeed a 64 bit version.

% file /usr/bin/ld '/usr/bin/ld: symbolic link to ld.lld' % file /usr/bin/ld.lld /usr/bin/ld.lld: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /libexec/ld-elf.so.1, for FreeBSD 12.0 (1200020), FreeBSD-style, stripped

So even the linker itself is aarch64, but it doesn't seem to accept the option '-m aarch64elf'. Or should this read '-m aarch64'?

I'm trying to find out more.

[brd]

@mburnicki The link you posted earlier is to the RPi2 image, so please post the full output of uname -a

[mburnicki]

@brd Sorry, the link I posted earlier was wrong. Quite some time ago I started with a Pi one, but then I saw there is no current image available now, so I thought about picking up and trying a Pi 2 or 3. Ended up with a Raspberry Pi 3 now, using the image FreeBSD-aarch64-12.0-GENERIC-313109M.img.

uname -a says: FreeBSD rpi3 12.0-CURRENT FreeBSD 12.0-CURRENT #0 r313109M: Thu Feb 2 16:16:39 MST 2017 raspberry@hive.raspbsd.org:/usr/home/brd/rpi3/crochet/work/obj/arm64.aarch64/usr/src/sys/GENERIC arm64

[mburnicki]

For those who are interested, here's an update what I've found out:

The package I'm actually trying to build is a driver package for Meinberg GPS and similar PCI cards. A previous version is available via ports, but there have been reports that this doesn't build for ARM: https://www.freshports.org/sysutils/mbgtools/

So I fixed compiler warnings and tried to build it on an ARM device, namely a Raspberry Pi, but the RaspBSD image was missing the source tree, so the kernel driver didn't compile, and hence my initial question here.

The current version of the driver package can be found here: https://www.meinberg.de/download/drivers/mbgtools-fbsd-dev-2017-05-05.tar.gz

My usual build environment has source tree of my driver package in a directory on my Linux workstation, which can be mounted on different target machines. Then I ssh into that target machine and see if the code builds fine (from a clean source tree, of course) in the mounted directory.

This worked well with standard PCs running FreeBSD 8, 9, 10, 11, and 12-CURRENT, and both the kernel modules and the user space programs run fine on those machines.

Also on the Raspi it builds fine, except for the linker error when building the kernel module: ld -m aarch64elf -d -warn-common -r -d -o mbgclock.kld mbgclock_main.o pcpsdrvr.o identdec.o rsrc_bsd.o unknown emulation: aarch64elf

There was no error when the user space tools were linked, but when I tried to run one of them I just got the message killed, and dmesg saying: pid 2352 (bash), uid 0, was killed: text file modification

On the other hand, a simple "hello world" program compiled on the Pi ran fine.

So I tried to find out what could cause the killed. My driver project uses recursive make calls, with a top level Makefile in the base directory, and in each subdirectory a tiny Makefile which just specifies the target binaries and object files to build it, and then includes the top level Makefile to provide the rules, common CFLAGs, etc.

So I created a "hello" project which contains both a simple Makefile and a recursive build: https://www.meinberg.de/download/temp/burnicki/make-test.tar.gz

The interesting thing is that both the simple and the recursive targets build fine if the source code is located in the local file system of the pi. However, if the example code is extracted on an NFS drive here, then only the simple "hello" works fine when run from the NFS drive, the "hello-1" and "hello-2" programs just report killed as mentioned above.

On the other hand, if I extract and build the driver package locally on the Pi's file system then the user space programs also run fine without being killed, but the linker error when linking the kernel driver still persists. I've no idea what could be the reason for this.

[brd]

Interesting, how would you connect the Meinberg card to the RPi? Obviously the RPi doesn't have a PCI slot for a GPS card. Or are you just using this as a test platform for getting the driver to compile so you can us it on one of the big arm64 servers like this: https://softiron.com/products/overdrive-3000/ ?

[rozmansi]

Are your problems compiling&linking on RPi3 random, or exactly repeatable?

FreeBSD-aarch64-12.0-GENERIC-313109M.img is inherently unstable producing frequent core dumps when compiling or stress-testing the RPi3. The https://svnweb.freebsd.org/base?view=revision&revision=313772 fixed the problem. Unfortunately, you have to compile newer FreeBSD image for RPi3 yourself since http://www.raspbsd.org/ is outdated.

[brd]

@rozmansi I am actually building a new image right now for RaspBSD :)

[mburnicki]

@brd Indeed I was just using the Pi as a test platform to see if my package builds on ARM without problems/warnings. Most of the code is commonly used for different operating systems including FreeBSD, Linux, Windows, and even plain old DOS. The kernel driver packages for Linux and windows also support Meinberg's USB devices, but USB support has not yet been ported to FreeBSD.

@rozmansi I tried the whole day to figure out under which conditions the errors occurred. From the example project https://www.meinberg.de/download/temp/burnicki/make-test.tar.gz

The "hello-simple" directory always builds and runs fine on the Pi. The "hello-recursive" directory also builds fine, but if the source code is stored on an NFS mount (the Linux PC in my case) then the compiled binaries "hello-1" and "hello-2" can't be executed. Instead, only the message killed is displayed, and afterwards dmesg says: pid 2352 (bash), uid 0, was killed: text file modification

This was actually exactly reproducible here, and is exactly what I observed with the user space programs from my driver package. For example, if you build the driver package and then run mbgstatus/mbgstatus a startup message is displayed, and the program says "No device found.", which is expected on the Pi. If I build the package from the same source code on my NFS drive then I just get killed when I try to run mbgstatus.

I've never seen before that it makes any difference whether I compile from an NFS drive or a local filesystem. I've also used the Raspi with Raspbian in the same way without problems to see if my driver package compiles fine under Linux/ARM.

The error when linking the kernel driver is quite a different thing, though. The same error occurs if I have the sources on the local file system. Eventually the problem is the linker binary itself. The RaspBSD page http://www.raspbsd.org/raspberrypi.html says it needs to use some packages from FreeBSD 11. I'm not familiar enough with FreeBSD to see if that matters.

[mburnicki]

@brd I'm going to try the new image when it's available.

[rozmansi]

@brd, I owe you a beer? :)

[brd]

Can one of you test the image I uploaded last night? I am on my way to OSCON and didn't have time to test it before leaving.

http://download.raspbsd.org/FreeBSD-aarch64-12.0-GENERIC-317984M.img.gz

[rozmansi]

@brd Here's what I got: rpi3-FreeBSD-aarch64-12.0-GENERIC-317984M.txt

Please note that countdown at "Hit [Enter] to boot immediately, or any other key for command prompt." was unusually slow - each "second" of the countdown took like five seconds. Is the clock configured correctly? The rest of the boot process was snappy. Until it hit the missing root FS. :(

[rozmansi]

@brd Here's the log of the previous RaspBSD version: rpi3-FreeBSD-aarch64-12.0-GENERIC-313109M.txt

I confirm this previous version has the unusually slow countdown I reported in my previous post too. Therefore, you can safely ignore my previous remark about clock configuration.

Another thing, I was trying to contact you about: How did you compile the r313109 to load the bcm2835_cpufreq0 device? When I took the stock r313109 out of SVN and use the Crochet to build the image using the default RPi3 setting, the bcm2835_cpufreq0 wouldn't load. The r317984 you prepared yesterday didn't load the bcm2835_cpufreq0 either. Needless to say, without the bcm2835_cpufreq0 the RPi3 does not dynamically adjust the CPU frequency and it feels a bit sluggish, idles warmer, draws more idle power etc.

I am aware we are getting off-topic here, but I promise I shall test compiling and linking stability ASAP I get a working RPi3 image.

[mburnicki]

Same problem here.

[brd]

OK, I pulled the image and will take a look this weekend once I get home. Sorry guys!

[brd]

@gvnn3 Do you have any insight on the Meinberg driver?

[okeozkilic]

@brd I have downloaded and tried the image file you have mentioned: http://download.raspbsd.org/FreeBSD-aarch64-12.0-GENERIC-317984M.img.gz

My Pine64+ 2GB did not boot with this image. I also tried the image 317973M and that did not boot either. The other two images boot my Pine64 (312006M and 308649M) without any problem.

I really want to build an image for Pine64 but I actually when I try to do that I got an error (tried both FreeBSD 11 and 12). Can you please help me with this problem?

I also opened a new thread for this Crochet build problem titled "FreeBSD 11 - Pine64 Build Problem". Any help will be truly appreciated.

[rozmansi]

@brd The issue with failed root-fs RW remount is present from around March. I never had time to track down what actually causes it.

Today, I found a workaround to boot RPI3 by going to single-user mode first, running mount -urw / followed by mount -a, then exit single-user mode to boot into multi-user mode.

Edit: Interesting, after the next boot the RPI3 boots just fine. I am getting an impression growfs might be the cause of root-fs RW remount issue.