flake: add bpir3_cross package

[steveej]

i've been running this kernel configuration since sunday in the absent of any unprecedented issues.

my x86_64-linux machine is more powerful than any of my aarch64-linux machines, hence i prefer cross compiling.

the change i'm least confident about is the kernel configuration. i'm also puzzled why the btrfs-subvol filesystem structure isn't working for me, OTOH it's also not a priority for me.

happy to receive any and all feedback on this :blush:

[nakato]

There's three changes in this single PR.

First, these two

* 25f1b4c - feat(bpir3): bump kernel to 6.9
* 2ab4489 - chore: bump nixpkgs
* 48a5868 - chore: add gitignore

They had a couple of nits from me, but rather than pollute this PR and detract from the final commit for cross compilation I've pulled them in, and fixed the nits I had.

I updated the commit messages on all three of those. I don't use "chore" "feat", etc, style. Commit messages should look similar to the OpenStack or the Kernel guidelines. There's not much to say about version bumps, so I only dropped the style parts, but I felt more could be elaborated on for the gitignore addition.

For the kernel version bump I added a commit before it to fix the typo on kernelPackages everywhere it occurred, and then fixed the reference to linuxPackages_6_9_bananaPiR3 in pkgs/default.nix and changed DRM.tristate = lib.mkforce no to DRM = lib.mkForce no. Setting the tristate attribute to null was resulting DRM still being a module.

Thanks for these updates!

---

my x86_64-linux machine is more powerful than any of my aarch64-linux machines, hence i prefer cross compiling.

Well, you aren't kidding! Compiling the kernel on my desktop took 18 minutes. My OPi5 is nowhere near that fast.

To update this PR, rebase it on main and drop the first three commits as they're in main now.

[nakato]

Probably the best way to get cross compilation working is going to be to not build fip.bin in ATF but instead do that during the u-boot build process, which is how every other system builds their firmware in nixpkgs.

I'm not sure how to do that at this time, which is why it's backwards.

Well, past me knew how to do that, there was even a comment explaining it. Shame I didn't see that before working it out again from scratch.

If you base your cross-compilation work on top of the ATFBeforeUboot branch here it should come together pretty easily now. I think it's just a matter of the correct HOSTCC as you've done and updating some references to use buildPackages.NAME to have them use the host platform and not the target for those tools.

[steveej]

i tested cross-compiling again on the most recent commit i force-pushed, and it worked. i built using

nix build -j0 -vL .\#nixosConfigurations.bananapi-bpir3_cross.config.system.build.sdImage

and i've got no aarch64 builders configured:

$ rg aarch /etc/nix/machines | wc -l
0

[tbaumann]

If I add

nixpkgs.buildPlatform.system = "x86_64-linux";
nixpkgs.hostPlatform.system = "aarch64-linux";

I get

› nix build ".#nixosConfigurations.hostname.config.system.build.sdImage"
trace: evaluation warning: system.stateVersion is not set, defaulting to 24.11. Read why this matters on https://nixos.org/manual/nixos/stable/options.html#opt-system.stateVersion.
error: builder for '/nix/store/jfhpdzjp6lcfw6x3rrairs47g5xn84ap-texinfo-interactive-aarch64-unknown-linux-gnu-7.1.drv' failed with exit code 1;
       last 10 log lines:
       > checking whether the compiler supports the __inline keyword... yes
       > checking for pthread_rwlock_t... yes
       > checking whether pthread_rwlock_rdlock prefers a writer to a reader... guessing no
       > checking for nl_langinfo... yes
       > checking whether YESEXPR works... guessing yes
       > checking whether wcrtomb works in the C locale... guessing yes
       > checking whether wcrtomb return value is correct... guessing yes
       > checking whether iconv converts EUC-CN to UTF-8... configure: error: in `/build/texinfo-7.1':
       > configure: error: cannot run test program while cross compiling
       > See `config.log' for more details
       For full logs, run 'nix log /nix/store/jfhpdzjp6lcfw6x3rrairs47g5xn84ap-texinfo-interactive-aarch64-unknown-linux-gnu-7.1.drv'.
error: 1 dependencies of derivation '/nix/store/nh8ncp9ivxay8h96z97sjfqywhkrmcx2-system-path.drv' failed to build
error: 1 dependencies of derivation '/nix/store/cpqdwpr9a794s30d1aq6ak41hn89a80q-nixos-system-nixos-24.11.20240802.d049530.drv' failed to build
error: 1 dependencies of derivation '/nix/store/pgqcxaj6x94q3j7sgkbrg2ql5lnl7wck-btrfs-fs.img-aarch64-unknown-linux-gnu.drv' failed to build
error: 1 dependencies of derivation '/nix/store/739fsnm0ph26x85xkqhmsj1i46wp3l41-nixos-sd-BananaPi-BPiR3-aarch64-unknown-linux-gnu.drv' failed to build

[tbaumann]

Probably https://github.com/NixOS/nixpkgs/issues/331204

[tbaumann]

branch nixos-24.05 works. Rebuilding half the things was faster than the qemu build for just the kernel package

[nakato]

Re: the texinfo failure.

The reason I've been dragging my feet with the idea of cross-build images in the main branch flake outputs is I hit a number of build failures when attempting them myself. Most of them were minor things, but when gitFull failed cryptically about perl support, and it coming down to perlSupport ? stdenv.buildPlatform == stdenv.hostPlatform in https://github.com/NixOS/nixpkgs/blob/ec0b407e87d185e63a853392555ffac756eaf6fb/pkgs/applications/version-management/git/default.nix#L8 that was when I threw my hands up.

branch nixos-24.05 works. Rebuilding half the things was faster than the qemu build for just the kernel package

I've been working on automating builds that go to cachix to reduce the build pain, but that means you have to have buildPlatform as aarch64-linux.

As for cross building or not, I can see pros and cons both ways.

Building on x86_64 means not needing to keep a copy of your repo synced to another machine, or setting up remote-builds, and it might be faster depending on the x86_64 machine. But, tests are skipped and you have to build the world every update. If you ever run nix shell nixpkgs#something on the aarch64 system, then you have to wait for a full system to download and unpack from cache.

On aarch64 you get the cache, but you need to either setup remote-builds, or you have to copy your repo around, which is a pain if you haven't committed your change yet.

So that brings me to a question, if the kernel was cached, would building with buildPlatform == hostPlatform be an option you would use?

[jhol]

I just published my attempt at this feature: https://github.com/nakato/nixos-sbc/pull/16

It extends the concept of this PR to SD-image packages in the flake for all boards.

The reason I've been dragging my feet with the idea of cross-build images in the main branch flake outputs is I hit a number of build failures when attempting them myself.

I understand your concern. For myself, it is far less practical to be limited to building images on AArch64 devices. I want to be able to cross-build SD images from my workstation PC, and I am willing to risk/trust that I won't be let down by uncaught test failures.

With this I built an image for my BPi-R4, and it seems to run ok :shrug: .

[nakato]

I've merged #16, which provides cross-build bootstrap images for all boards.

Documented at https://github.com/nakato/nixos-sbc?tab=readme-ov-file#bootstrap-images

[steveej]

I've merged #16, which provides cross-build bootstrap images for all boards.

great, thank you! glad to see that have landed.