Update to 12.4 with clang

michael-o commented 3 years ago

Can this be updated to 12.4 with clang from base? 9.3 is dead for many many years and GCC is not the standard compiler anymore in base.

MrDOS commented 2 years ago

I'll take a look. I suspect the process of building Clang is fairly different from building GCC, so it may take me some time to figure out.

MrDOS commented 1 year ago

I've rebuilt this image around the native cross-compilation capability in Clang/LLVM, and just pushed a new version targeting FreeBSD 11.4. My preference is to target a reasonably old OS version, as my own use case for the container is building the native portion of a Java serial communications library, and I think it's important to have low minimum requirements for a library that may be used in some very low-churn environments. Is there anything in particular that you wanted from FreeBSD 12 that isn't in 11? (Or have I misunderstood some aspect of FreeBSD's backwards compatibility, and should I be thinking about FreeBSD targets differently?)

It may also be worth noting that this new version represents a departure from my previous tooling philosophy. I went to great lengths when targeting 9.3 to match the compiler version in the image to the compiler version that was distributed with the OS. I haven't done that this time; this new image uses the latest and greatest Clang (16) packaged by Alpine. IMO that's probably more useful than a newer base system target.

michael-o commented 1 year ago

Good reasoning, this image is also used for Jansi which is used in Maven, that is why I care as a Maven committer. I am totally fine with 11.4 and support your position as long as the following is met:

Consumers don't need to install compat11 if some library version has been bumped
The updated clang does not require any runtime library dependency which is not part of the 11+ base system. I know this when using GCC from ports, you must add it to RPATH otherwise your executables won't do.

then this issue is satisfied.

Is there any particular reason why you need latest clang and the base compiler won't do?

Reported with Jansi: https://github.com/fusesource/jansi/issues/257

PS: Thank you for the huge effort!

MrDOS commented 1 year ago

Thank you for spelling out your prerequisites. I think both of them apply to me, too – I just don't know enough about FreeBSD to have been able to articulate them with such specificity. (I don't use FreeBSD, personally, so I find myself a bit off-kilter testing this project.)

My understanding from little research is that FreeBSD supports binary compatibility with itself at two different levels: at the syscall level, and at the library level for select system libraries.

For syscalls, the kernel defines a series of compatibility options for previous major versions. These are all enabled for GENERIC amd64 and i386 kernels; the arm64 kernel, which has only existed since FreeBSD 11, obviously has syscall compatibility only with FreeBSD 11 and 12. This lets the kernel implement the same syscall multiple times to extend compatibility (e.g., standard fstat vs. FreeBSD 11 fstat). However, for typical applications and libraries, this shouldn't matter one way or the other, because syscalls will be made via the dynamically-linked libc, not by the application itself.
For system libraries, the “compat” packages you point out come into play. The compat11x pkg-plist suggests that you can avoid it as long as you don't depend on libarchive, libcasper, OpenSSL, Milter, “libproc” (Linux compatibility library?), ZFS, and the... ATM driver for SNMP? Really mixed bag. Definitely possible to accidentally walk into some of these, but I can't see either of our use-cases doing so.

As an aside, I found it really difficult to search for this stuff, as Google thinks you want to know about FreeBSD's compatibility with Linux binaries, not binaries built for older versions of FreeBSD!

I took this for a practical test by using this container to build ffmpeg, and confirmed that it ran on a clean install of FreeBSD 13.2 on arm64 with a GENERIC kernel and no compat11x port installed. I also built a shared library that invoked fstat (a syscall that changed between FreeBSD 11 and 12), and confirmed that, compiled for FreeBSD, it could be loaded by and called from an executable built on FreeBSD 13 (proving that libc works as a natural syscall compatibility layer, as expected).

Is there any particular reason why you need latest Clang and the base compiler won't do?

I don't need the latest Clang, per se, but because I'm now able to use a generic compiler built by someone else, it's more convenient to do that rather than build my own. Alpine 3.18 only packages Clang 14/15/16, while FreeBSD 11.4 included Clang 10, so I can't match even the major version; in which case, it's as easy to use the latest version as any other. I might look into compiling my own Clang, though; that would be one way to shed the useless inclusion of GCC and libstdc++ and to cut down the image size a bit.

Thanks a lot for your interest and feedback. If this all adds up to you then I'll close this issue, but I'm also open to the possibility of generating separate images to support FreeBSD 12/13 as long as they'll get used. While I haven't tried it, I suspect the current Dockerfile would also be able to target those OSes without modification when built with an appropriate FBSD_VERSION value.

michael-o commented 1 year ago

Let me get back to you next week. You wrote a high-quality answer, I want to reply to it decently!

michael-o commented 1 year ago

While I havn't yet processed the reply yet fully, I see already the first problem:

[osipovmi@al8 sqlite-jdbc]$ docker run empterdose/freebsd-cross-build:11.4 which i386-freebsd11-strip

[osipovmi@al8 sqlite-jdbc]$
[osipovmi@al8 sqlite-jdbc]$ docker run empterdose/freebsd-cross-build:9.3 which i386-freebsd9-strip
/usr/local/bin/i386-freebsd9-strip

There is no qualified strip in this image.

Output:

9 warnings generated.
i386-freebsd11-clang -Itarget/sqlite-3.43.0-FreeBSD-x86 -Itarget/sqlite-amalgamation-3430000 -I/include -Ilib/inc_linux -Os -fPIC -fvisibility=hidden   -I target/common-lib -c -o target/sqlite-3.43.0-FreeBSD-x86/NativeDB.o src/main/java/org/sqlite/core/NativeDB.c
clang-16: warning: argument unused during compilation: '-fuse-ld=lld' [-Wunused-command-line-argument]
i386-freebsd11-clang -Itarget/sqlite-3.43.0-FreeBSD-x86 -Itarget/sqlite-amalgamation-3430000 -I/include -Ilib/inc_linux -Os -fPIC -fvisibility=hidden   -o target/sqlite-3.43.0-FreeBSD-x86/libsqlitejdbc.so target/sqlite-3.43.0-FreeBSD-x86/NativeDB.o target/sqlite-3.43.0-FreeBSD-x86/sqlite3.o -shared
cp target/sqlite-3.43.0-FreeBSD-x86/libsqlitejdbc.so /tmp/libsqlitejdbc.so
i386-freebsd11-strip /tmp/libsqlitejdbc.so
make: i386-freebsd11-strip: No such file or directory
make: *** [Makefile:110: target/sqlite-3.43.0-FreeBSD-x86/libsqlitejdbc.so] Error 127
make: *** [Makefile:148: freebsd32] Fehler 2
[osipovmi@al8 sqlite-jdbc]$ vim Makefile
[osipovmi@al8 sqlite-jdbc]$ docker -it empterdose/freebsd-cross-build:11.4 bash

The bundled strip as well as the other tools which are missing aren't usable on FreeBSD binaries, of course.

MrDOS commented 1 year ago

Ah, you're right, I didn't install the full llvm16 package in the container. I've opened #2 to track that.

I think, if I'm going to have to go down this road anyway, then I might try to do a custom build of LLVM/Clang after all. Mostly so that I can keep the size of the image down by avoiding the inclusion of target-incompatible Gnu Binutils.

michael-o commented 1 year ago

The first success after local hacks:

osipovmi@deblndw011x:~/var/Projekte/sqlite-jdbc (master *%=)
$ ~/.local/bin/diffoscope --debug /tmp/libsqlitejdbc.so target/sqlite-3.43.0-FreeBSD-x86_64/libsqlitejdbc.so
2023-09-11 17:04:59 D: diffoscope.main: Starting diffoscope 247
2023-09-11 17:04:59 D: diffoscope.presenters.formats: Will generate the following presenter formats: text
2023-09-11 17:04:59 I: diffoscope.main: Fuzzy-matching is currently disabled as the "tlsh" module is unavailable.
2023-09-11 17:04:59 D: diffoscope.environ: Normalising locale, timezone, etc. PATH is /sbin:/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/net/home/osipovmi/bin
2023-09-11 15:04:59 D: diffoscope.main: Starting comparison
2023-09-11 15:04:59 D: diffoscope.comparators.decompile: radare2 not found, disabling decompiler
2023-09-11 15:04:59 D: diffoscope.comparators: Loaded 90 comparator classes
2023-09-11 15:04:59 D: diffoscope.comparators.utils.specialize: Using elf.ElfFile for /tmp/libsqlitejdbc.so
2023-09-11 15:04:59 D: diffoscope.comparators.utils.specialize: Using elf.ElfFile for target/sqlite-3.43.0-FreeBSD-x86_64/libsqlitejdbc.so
2023-09-11 15:04:59 D: diffoscope.comparators.utils.compare: Comparing /tmp/libsqlitejdbc.so (ElfFile) and target/sqlite-3.43.0-FreeBSD-x86_64/libsqlitejdbc.so (ElfFile)
2023-09-11 15:04:59 D: diffoscope.comparators.utils.file: has_same_content(/tmp/libsqlitejdbc.so, target/sqlite-3.43.0-FreeBSD-x86_64/libsqlitejdbc.so)
2023-09-11 15:04:59 D: diffoscope.comparators.utils.file: Executing: cmp -s /tmp/libsqlitejdbc.so target/sqlite-3.43.0-FreeBSD-x86_64/libsqlitejdbc.so
2023-09-11 15:04:59 D: diffoscope.comparators.utils.compare: has_same_content_as returned True; skipping further comparisons
2023-09-11 15:04:59 D: root: # Profiling output for: /net/home/osipovmi/.local/bin/diffoscope --debug /tmp/libsqlitejdbc.so target/sqlite-3.43.0-FreeBSD-x86_64/libsqlitejdbc.so
2023-09-11 15:04:59 D: root: ## command (total time: 0.024s)
2023-09-11 15:04:59 D: root:        0.012s      1 call     cmp
2023-09-11 15:04:59 D: root:        0.012s      1 call     cmp (external)
2023-09-11 15:04:59 D: root:        0.000s      1 call     cmp (internal)
2023-09-11 15:04:59 D: root: ## has_same_content_as (total time: 0.013s)
2023-09-11 15:04:59 D: root:        0.013s      1 call     abc.ElfFile
2023-09-11 15:04:59 D: root: ## main (total time: 0.198s)
2023-09-11 15:04:59 D: root:        0.198s      2 calls    outputs
2023-09-11 15:04:59 D: root:        0.000s      1 call     cleanup
2023-09-11 15:04:59 D: root: ## recognizes (total time: 0.006s)
2023-09-11 15:04:59 D: root:        0.005s     60 calls    diffoscope.comparators.binary.FilesystemFile
2023-09-11 15:04:59 D: root:        0.001s     58 calls    abc.ElfFile
2023-09-11 15:04:59 D: root: ## specialize (total time: 0.002s)
2023-09-11 15:04:59 D: root:        0.002s      1 call     specialize

Left shared object produced on AL8 with your cross image, right compiled natively on FreeBSD 12-STABLE. At first glance, very nice. I am waiting for the fix in #2.

MrDOS / freebsd-cross-build

Update to 12.4 with clang #1