Questions and Bugs

[rodw-au]

When building the installer on Bookworm X86 host, I got this silly error which displays in a window Building for a Pi 4b 64 bit The file displayed exists in the root folder complete with the -- prefix

First encountered doing a make all from make menu Reran again to build a patched preempt_rt kernel (no other changes except adding a patch and changing menuconfig to fully preemptible kernel) and it happened again. I noticed the kernel was built so I tried selecting make bootable file system and the error was immediately displayed again.

Deleted the file and the error persisted.

I don't know how to debug make here. Help would be appreciated.

[pyavitz]

I'll investigate. In the mean time you can also build using make commands

Is it actually producing an img? If so this may just be a dialog error.

[pyavitz]

I just ran native and got this.

That was building for the rootfs only. I have an x86_64 build currently going on all. I'll get back to you.

[rodw-au]

Thanks for a prompt response. The first time it did build an image, second time it didn't. I just did this and it seems to just print a file name: debian@debian:~/devel/rpi-img-builder-lcnc$ make rootfs board=bcm2711 Root Filesystem 256968 -rw-r--r-- 1 root root 263120456 Aug 27 08:45 debian-bookworm-rootfs-arm64.tar.xz

But that file date is too early for it to be my RT build and I would have expected to see RT in the file name

[rodw-au]

So I might start again with: make purge-all make all board=bcm27111 and report back in a few hours

[pyavitz]

Nah. It is just telling you the rootfs tarball is already there. I have the builder setup, so if it finds the tarball, it skips the build process. Since it is time and bandwidth consuming.

If you wanna really purge everything, you need to remove the tarball ur self. sudo rm -f *tar*

[rodw-au]

NOw its really crazy. when I say purge all, it starts to build the kernel! I might reclone

[rodw-au]

Recloned. Here is another bug. When you set MENUCONFIG=1 in build config, the data is wrong. You get this and an error later and menuconfig does not load.

MENUCONFIG="1"01

[rodw-au]

Cloned again so Brand new Does this look right? (loaded su so sudo does not time out) /install.sh make config sudo su make all board=bcm2711

RT patch from kernel.org in userpatches deleted all kernel deb files from /output

[pyavitz]

Are you back spacing? dialog is a bit silly. So you need to make sure you delete the values listed before entering new ones.

Came right up for me?

[pyavitz]

Cloned again so Brand new Does this look right? (loaded su so sudo does not time out) /install.sh make config sudo su make all board=bcm2711

RT patch from kernel.org in userpatches deleted all kernel deb files from /output

Yes, looks good to me.

[rodw-au]

Thanks, building now. The patches seemed to be applied and menuconfig came up. I am hoping to fork this for linuxcnc so they can put it in their buildbot. In the past they used a Pi which failed about once a month. Will be compiling linuxcnc from source. I think I know how to make it do that.

The dialog actually caught me out every time so I check it in a text editor to get it right.

[pyavitz]

I'm thinking that menuconfig bit might have something to do with this piece of code.

# userdata-txt
if [[ -f "userdata.txt" ]]; then
    sed -i 's/=0/="0"/g' userdata.txt
    sed -i 's/=1/="1"/g' userdata.txt
    sed -i 's/LLVM_IAS="1"/LLVM_IAS=1/g' userdata.txt
    echo -e "userdata file created."
    exit 0
fi

It doesn't know what you entered and is just putting quotes around what it is looking for.

[rodw-au]

It actually flagged it as invalid data and said it was ignoring it. First time through, I had no idea what that meant :)

[pyavitz]

I just saw what you wrote. And I'm not sure how make purge-all would try and build anything?

Craziness. I even tried to make it happen, which shouldn't work and didn't.

[rodw-au]

I just saw what you wrote. And I'm not sure how make purge-all would try and build anything?

Craziness. I even tried to make it happen, which shouldn't work and didn't.

Yes, I think by then, it was totally screwed! I must have broken it.... Anyway, a fresh start now I know what I am doing should work... :)

[rodw-au]

Hmm, adduser: Please enter a username matching the regular expression configured via the NAME_REGEX configuration variable. Use the `--allow-bad-names' option to relax this check or reconfigure NAME_REGEX in configuration. user and PW were both CNC in the userdata.txt

[rodw-au]

OK, must all be lower case..I didn't know that. Ubuntu only accepts usernames based on the following rules: Must start with a lowercase letter May only contain lowercase letters, underscore (_), and dash (-) May optionally end with a dollar sign ($)

So I changed the userdata.txt to my name and reran
make image board=bcm2711 and I got: adduser: The user `rodwebster' does not exist. make: *** [Makefile:136: image] Error 1

[pyavitz]

Hmm, adduser: Please enter a username matching the regular expression configured via the NAME_REGEX configuration variable. Use the `--allow-bad-names' option to relax this check or reconfigure NAME_REGEX in configuration. user and PW were both CNC in the userdata.txt

That is interesting, I also didn't know this. I normally don't get complicated with my user information though.

This worked fine for me.

NAME="CNC"
USERNAME="cnc"
PASSWORD="cnc"

Last I checked the only variable you really need to be careful with here is NAME="".

GOOD

NAME="Marvin"

FAIL

NAME="Marvin Martian"

It doesn't like blank spaces.

[rodw-au]

Thanks I finally got success! I set name = user = password = cnc I won't get a chance to test it until tomorrow but I have every confidence. :)

[rodw-au]

Finally was able to attempt to boot this image. I got a fail as shown below. I figured the boot loader might be old so upgraded it but still got the same error. It appears to not be able to find a FAT32 boot partition. I noticed in custom.txt there is the following setting `

Root Filesystem Types: ext4 btrfs xfs

FSTYPE="ext4" ` But I don't see an option to use FAT. (Is that actually for the the Boot partition?)

Have you got any ideas here? Using a Pi 4B about 4 years old (pre covid) and a 15gb SD card. I'm a bit lost on how to resolve this. I am a very occasional Pi user...

[rodw-au]

That wasn't it. There is a FAT16 Boot partition

mmcblk0p1 vfat FAT16 BOOT 0783-4AD5 208.1M 17% /media/debian/BOOT

[pyavitz]

I no longer own a Pi4, but I wonder if me turning off LBA has in some way created an issue?

Let us check and see. You'll need a Linux box for this. Put the SD in the CARD reader and plug it in to ur linux box. Now find the /dev/$node and run: sudo parted --script /dev/$node set 1 lba on

Example:

sudo parted --script /dev/sdd set 1 lba on

You can check and make sure if you like.

sudo fdisk -l /dev/sdd
Disk /dev/sdd: 29.72 GiB, 31914983424 bytes, 62333952 sectors
Disk model: MassStorageClass
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xd29ca7e4

Device     Boot  Start      End  Sectors  Size Id Type
/dev/sdd1         2048   526335   524288  256M  c W95 FAT32 (LBA)
/dev/sdd2       526336 62332927 61806592 29.5G 83 Linux

[pyavitz]

Hmm. Apparently this is an issue; https://forums.raspberrypi.com/viewtopic.php?t=278295

I'll investigate. Thanx for the report.

[pyavitz]

Sorry about all that. This should resolve it: https://github.com/pyavitz/rpi-img-builder/commit/f3f48ad3301fe7a8be05e1081380e04aab6a0802

[rodw-au]

A very interesting article even if its annoying! I might try your LBA tweak, then erase the MBR this way https://www.cyberciti.biz/faq/linux-clearing-out-master-boot-record-dd-command/ I do remember walking a 12 bit FAT 30 years ago but never again! Seems like I should be trying to write a standard Rpi image to start with in the hope it corrects the MBR

[rodw-au]

Thanks. Just going to bed. so will try tomorrow night.

[rodw-au]

It worked! PREEMPT_RT kernel installed perfectly! XFCE installed LinuxCNC dependencies installed (about 2 Gb) Building LinuxCNC now. That will take a while.... Once I have it all down pat, it will be time to script it in the installer. Thanks for all your help. I really appreciate the prompt response and guidance. Perhaps leave the issue open for a few days and I'll report any other issues..

[pyavitz]

Excellent! I'm happy we got it figured out.

It appears in my quest to make things better, I broke something, which isn't unusual. :) Thanx for testing and helping to resolve it.

[rodw-au]

I break things all the time! Is there a folder location you can add debs to that are installed when it runs? That would be a useful shortcut to start with. for me,

[pyavitz]

Not a particular location, no. There is a USER_PKGS variable in the user data file you can add your own list of packages to. But beyond that there is only the userscripts.

Using it requires you know bash. Basically you need to create a function or functions and the builder will run them. You'll also need to enable USCRIPTS="1" in the user data file.

The README gives a "basic" run down of how it works, along with some example scripts I've made and used. Two examples to look at, would probs be util-linux and gpac.

If you are adding a large amount of packages, I suggest increasing the size of the img. This can be done via the custom.txt file.

[rodw-au]

Somehow I am getting this error. I did not see it previously. I skipped past it. Is it a critical patch?

`-------------------------- |diff --git a/net/core/devlink.c b/net/core/devlink.c |index 5a4a4b34ac15..f0c6fcdb7a3a 100644 |--- a/net/core/devlink.c |+++ b/net/core/devlink.c

File to patch: ` So I added a user script and it ran last night fine until it ran out of space right at the end (with 6 gb). Then I increased the size and accidentally added some spurious characters to uscript so it errored. Fixed that this morning, then got this error. I am also seeing it at home on a different PC. Odd.

[pyavitz]

It is a good idea to work with commits or at least know the commit you are working out of, when using RT patches. They are very tricky and revision specific. You can sometimes get away with going up a revision or two, but it isn't the best practice.

In your case I believe rpi-linux has taken to many steps forward to support the patch you are using. So use the commit feature of the builder: make commit board=bcm2711

Paste commit: 655fc658a15ae7a6f37103754adb39ba52a9a14e

[pyavitz]

As for the size, the builder only knows what we tell it. If you wanna play it safe and not math out the build, give it an exaggerated image size. I usually go with 8GB or 10GB and just let it shrink the img down when it is done.

[rodw-au]

Thanks, Its all a learning process. I looked at the kernel version in the beginning and it was 6.1.47 vs the RT patch at 6.1.46. I did not know how to change it back but I figured it would be close enough. I did not realise the Pi kernel is marching forward so its now up to 6.1.50!

Thanks for explaining how to get an earlier version.

[rodw-au]

Paste commit: 655fc658a15ae7a6f37103754adb39ba52a9a14e

Can you explain where you got this commit from. I tried using the commit fot Linux 6.1.46 but its different to this (and did not work)

[pyavitz]

Paste commit: 655fc658a15ae7a6f37103754adb39ba52a9a14e

Can you explain where you got this commit from. I tried using the commit fot Linux 6.1.46 but its different to this (and did not work)

Didn't apply correctly or the RT aspect of the kernel isn't working? I built the kernel to make sure it compiled but I never ran it.

[pyavitz]

I git cloned the repo and ran git log --merges -n 2 Which gave me the last two merge remote tracking tags. The foundation also keeps tags in their github under releases, but unfortunately the last tag is really old.

[pyavitz]

Ok. So this is what I just did:

marvin: rpi-linux git:( rpi-6.1.y ✓ ) $ git log --merges -n 4
commit 9b79cb06ad370cfd5fc52f7d85d82bdcff701828
Merge: 9d9586dc0c0d a2943d2d9a00
Author: Dom Cobley <popcornmix@gmail.com>
Date:   Thu Aug 31 18:58:24 2023 +0100

    Merge remote-tracking branch 'stable/linux-6.1.y' into rpi-6.1.y

commit 655fc658a15ae7a6f37103754adb39ba52a9a14e
Merge: e89e7655a197 802aacbbffe2
Author: Dom Cobley <popcornmix@gmail.com>
Date:   Fri Aug 25 13:04:30 2023 +0100

    Merge remote-tracking branch 'stable/linux-6.1.y' into rpi-6.1.y

commit 9ed4f05ba2e2bcd9065831674e97b2b1283e866d
Merge: 19a1b0352936 6c44e13dc284
Author: Dom Cobley <popcornmix@gmail.com>
Date:   Mon Aug 21 17:44:22 2023 +0100

    Merge remote-tracking branch 'stable/linux-6.1.y' into rpi-6.1.y

commit 19a1b03529363945fbbb4b9160fe8645809a9dce
Merge: 0917a6caec99 1321ab403b38
Author: Dom Cobley <popcornmix@gmail.com>
Date:   Fri Aug 11 14:39:56 2023 +0100

    Merge remote-tracking branch 'stable/linux-6.1.y' into rpi-6.1.y

This gives the last four they did. I then check the dates against kernel.org here: https://cdn.kernel.org/pub/linux/kernel/v6.x/

I see this: linux-6.1.46.tar.xz 16-Aug-2023 16:39 129M

Now assuming that logic still works, I go with the following commit: 9ed4f05ba2e2bcd9065831674e97b2b1283e866d

With that said, don't always assume logic is a thing when it comes to linux :) and there may be a better way to go about it? This is just the approach I am taking. If we were dealing with mainline and pulling from kernel.org this would be really straight forward.

[rodw-au]

Awesome. That makes sense too. Shown here in github

I'm a bit in between here as I am trying to set up a PC here at home (18 core i7) With Debian 12 on it to make this a bit faster... Just not sure if I'm going to overwrite a windows partition

I think you could aso search the log for "6.1.46" and compare with your merges. Its also complicated becasue the Pi does not track the mainline at kernel.org but matching up the RT patch version with mainline is always an issue to consider.

[rodw-au]

Finally got a working image with all our apps that matches the RT patch! Thanks for all your help so far. Don't ask me how long it takes to build! A couple of final questions.

1. After we have run menuconfig once, if we set MENUCONFIG="0", does that keep the kerenel config with preempt_rt set?
2. Can you change the code so the commit number can be stored in userdata.txt so we can use this in a buildbot situation? That would be awesome!

Some observations:

1. The user can use sudo without entering a password (we don't want to add a root user)
2. We are getting a locale error. I ended up changing the config to use my local Australia/Brisbane setting but I'm sure it is happens with the defaults. perl: warning: Setting locale failed. perl: warning: Please check that your locale settings: LANGUAGE = "en_AU:en", LC_ALL = (unset), LANG = "en_AU.UTF-8" are supported and installed on your system. Thanks again for being so helpful.

[pyavitz]

Finally got a working image with all our apps that matches the RT patch! Thanks for all your help so far. Don't ask me how long it takes to build! A couple of final questions.

1. After we have run menuconfig once, if we set MENUCONFIG="0", does that keep the kerenel config with preempt_rt set?

No it doesn't. That feature just allows you to run the interface and save the defconfig. Once you have a defconfig you like you can use the custom_defconfig feature in the userdata.txt. https://github.com/pyavitz/rpi-img-builder#user-defconfig

### CUSTOM
CUSTOM_DEFCONFIG="1"
MYCONFIG="cnc_defconfig"

2. Can you change the code so the commit number can be stored in userdata.txt so we can use this in a buildbot situation?

I could do this or something equivalent to it.

That would be awesome!

Some observations:

1. The user can use sudo without entering a password (we don't want to add a root user)

This can currently be disabled using menu-config or you can manually delete the file sudo rm -f /etc/sudoers.d/010_*-nopasswd. I suppose I could also add a switch to have this feature disabled by default during img creation?

2. We are getting a locale error. I ended up changing the config to use my local Australia/Brisbane setting but I'm sure it is happens with the defaults. perl: warning: Setting locale failed. perl: warning: Please check that your locale settings: LANGUAGE = "en_AU:en", LC_ALL = (unset), LANG = "en_AU.UTF-8" are supported and installed on your system.

After firstboot you either need to run setup -r or use menu-config to set the locales, timezone and other bits.

Thanks again for being so helpful.

You are welcome.

[rodw-au]

Well, I think its about done.

The one thing I forgot to ask was how to add boot parameters the right way. I need to add processor.max_cstate=1 isolcpus=2,3

The new commit method works perfectly. Much appreciated as we can now set and forget it!

I deleted the sudoers no password files in my stage 2 uscript. (I hope!)

So just to clarify re defconfig. I ran it once with menuconfig and copied the generated defconfig file to /defconfig and renamed it. Seems to be working.

Just getting to the end of another image build now.

[pyavitz]

Well, I think its about done.

The one thing I forgot to ask was how to add boot parameters the right way. I need to add processor.max_cstate=1 isolcpus=2,3

The new commit method works perfectly. Much appreciated as we can now set and forget it!

I deleted the sudoers no password files in my stage 2 uscript. (I hope!)

So just to clarify re defconfig. I ran it once with menuconfig and copied the generated defconfig file to /defconfig and renamed it. Seems to be working.

Just getting to the end of another image build now.

We can go about this ~two~ three ways.

• 1) Edit the lib/boards/bcm2711 file and add to variable EXTRA="" under # cmdline.
• 2) Edit the /boot/cmdline.txt file using uscript.

if [[ -f "/boot/cmdline.txt" ]]; then
    sed -i 's/net.ifnames=0/net.ifnames=0 processor.max_cstate=1 isolcpus=2,3/g' /boot/cmdline.txt
fi

I would go with option one, as the builder will sometimes generate a new /boot/cmdline.txt file under certain circumstances, as does the firstboot script. Putting it in the $BOARD file will insure it will always be there, as that file gets installed on the img at /etc/opt/board.txt.

EDIT: Actually there is a third option that should also work. Add to uscript a run_function0:

run_function0 (){
if [[ -f "board.txt" ]]; then
    sed -i 's/net.ifnames=0/net.ifnames=0 processor.max_cstate=1 isolcpus=2,3/g' board.txt
fi
}

That should edit the file before it ever gets sourced and used by the builder and install the edited file on the img.

[rodw-au]

Thanks. Option 1 seemed to be the cleanest way anyway but function0 seems safer again!

We seem to be wasting some space with redundant kernels and headers Selecting previously unselected package linux-headers-bcm2711-rpi-4. (Reading database ... 168835 files and directories currently installed.) Preparing to unpack linux-headers-bcm2711-rpi-4_6.1.50-rt13-1_arm64.deb ... Unpacking linux-headers-bcm2711-rpi-4 (6.1.50-rt13-1) ... Selecting previously unselected package linux-image-bcm2711-rpi-4. Preparing to unpack linux-image-bcm2711-rpi-4_6.1.50-rt13-1_arm64.deb ... Unpacking linux-image-bcm2711-rpi-4 (6.1.50-rt13-1) ... Setting up linux-headers-bcm2711-rpi-4 (6.1.50-rt13-1) ... Compiling headers ... Setting up linux-image-bcm2711-rpi-4 (6.1.50-rt13-1) ... update-initramfs: Generating /boot/initrd.img-6.1.50-rt13 n I have seen some people suggest to use rm to remove them after the one you want is installed. Any thoughts on that? Not sure how to do it though!

[pyavitz]

If you don't need the headers, rm -f /root/linux-headers-*.deb using uscripts. Probs the easier way, as the builder won't even run stage1 without running a check to make sure they are there. Just know, you won't be able to compile out-of-tree drivers with out them.

[rodw-au]

Its OK. We need headers for some out of tree stuff but its offically not something we can include due to obscure licensing so I will just add their repos which I do in my x86 installer.

Just did a final commit to my fork! Well hope its final!

[rodw-au]

Oops, whats this?

debian@debian:~/devel/rpi-img-builder-lcnc$ make commit kernel board=bcm2711
Compiling kernel
board.txt: line 36: unexpected EOF while looking for matching `"'

[rodw-au]

board.txt

[pyavitz]

Oops, whats this?

debian@debian:~/devel/rpi-img-builder-lcnc$ make commit kernel board=bcm2711
Compiling kernel
board.txt: line 36: unexpected EOF while looking for matching `"'

You are missing a quote.

EXTRA="loglevel=5 net.ifnames=0 firmware_class.path=/lib/firmware/updates/brcm

Should be.

EXTRA="loglevel=5 net.ifnames=0 firmware_class.path=/lib/firmware/updates/brcm"

[rodw-au]

Ahh Its being overridden by the function0 script. At least we know that works! Not sure on sed. Just used it on this project for the first time