TheSin- / rpi-img-builder

Scripts to create custom images for RaspberryPi
MIT License
107 stars 30 forks source link

resizing root partition leads to a kernel panic #8

Closed mzemb closed 7 years ago

mzemb commented 7 years ago

Hi.

Thanks for you software, I had in mind to write something like that and you saved me a lot of time. :)

That being said, I have a small problem: I'm creating an image for a raspberry compute module 2 (which is basically the rpi3 and associated flash instead of the sd card), but after the boot after the resize2fs leads to a kernel panic.

As a workaround, I've disabled the resizefs module and to set the size to my flash size in the common.mk, but I wonder what the problem is. (And it would be more elegant and practical to carry around smaller image files).

Note that calling manually fdisk and resize2fs leads to the same result, and that the official jessie lite image works like a charm.

Do you have any pointers for me ?

Regards,

M.

TheSin- commented 7 years ago

that is odd I wrote this for the CM3, So you are using the CM3 not the CM3 Lite then I assume since you mentioned emmc.

I've been doing most of my testing with the CM3Lite since SD is easier for faster tests. I'll see if I can find time today to test it with the full CM3. Can I have the exact line you used to build your image to make sure I'm using the same kernel and such, as I normally use the rpi-next plugin, specially for CM3 stuff.

I'll also check the current raspbian see if they made changes to their resize script though off the top of my head I have no idea why it wouldn't work, as you stated it's just fdisk and resize2fs. I might not get to it today but I'll try hard to if not sometime this week I should be able to look into it.

mzemb commented 7 years ago

I'm indeed using the CM3 with emmc.

I'm simply calling sudo make from the root of your repo (which is on your latest commit c9f274e from 15 feb 2017).

However I changed the common.mk with following patch: RPI ?= 3 UPASS ?= xxx RPASS ?= xxx UBOOT_DIR := rpi_3

I desactivated the hostname plugin, but I don't see it making a difference.

For completeness, I should mention that I've also mirrored locally the raspbian repository (through apt-mirror), to be able to fully reproduce the image without variation in time. I'll be surprised to be the cause of my problem, but I'll try with the online repo anyway.

I'm not using rpi-next as it was in disabled. I'm correct to assume it's the script from https://github.com/Hexxeh/rpi-update which update the kernel ? I'll try that first.

TheSin- commented 7 years ago

you don't need to modify common.mk just call it like

make RPI=3 UPASS=xxx RPASS=xxx

the UBOOT_DIR won't matter for raspbian as it doesn't use UBOOT, and the RPI 3 will only add the wifi module which the CM3 doesn't have anyhow. But the point is that it can all be done via command line making it cleaner to do a git fetch/pull on in the future ;)

yeah deactivating the hostname won't hurt, but if you include a hostname file it won't even run so either way it's fine.

It isn't rpi-update exactly, it installs the script then runs it to update to 4.9.x and the latest firmware rather then the current raspbian release version. Last I tried it was required to get the CM3 dtb, as it wasn't in raspbian yet.

mzemb commented 7 years ago

Well, my try without my raspbian mirror and including the rpi-update plugin still show the same behaviour: on the beginning I get into the system which works and shows the full size of the mmcblk0p2 partition, but I cannot reboot since after that I'm stuck with a kernel panic.

I have a CM3 on a CM1 dev board, but I doubt it's an issue.

TheSin- commented 7 years ago

is there any way to get the kernel panic you are seeing? if it's full size then it's past resize it might be an other plugin, also before you reboot the last time can you tell me the contents of /var/tmp please?

mzemb commented 7 years ago

Sorry for the delay. I'll try to give you as many information as possible.

I created a new image to investigate the problem. Still from your commit c9f274e, I just activated the rpi-next plugin, and I changed the source from http://mirrordirector.raspbian.org/raspbian to http://archive.raspbian.org/raspbian in the repos/Raspbian/multistrap.list.in file (I was having issue with the raspi42.fr mirror).

I used:

sudo make RPI=3 UPASS=raspberry RPASS=raspberry >make.log 2>&1 && echo "ok"

Note that although it completes successfully, make is throwing multiples messages "sh: 1: udevadm: not found", I wonder if it can be linked to our current issue.

Once logged into the CM, before I reboot into the kernel panick, /var/tmp contains only an empty file 'system-resizefs' .

It's hard to get the exact kernel output because it's fast and I don't have an uart console, but I managed to get following snippet of interest (manually copied from a few camera shots, be aware of possible typos...):

Waiting for root device /dev/mmcblk0p2...
mmc0: new high speed MMC card at address 0001
mmcblk0: mmc0:001 4FPD3R 3.64 GiB
mmcblk0boot0: mmc0:0001 4FPD3R parition 1 4.00 MiB
mmcblk0boot1: mmc0:0001 4FPD3R parition 2 4.00 MiB
mmcblk0rpmb : mmc0:0001 4FPD3R parition 3 512 KiB
random: fast init done
 mmcblk0: p1 p2
usb 1-1: New USB device found, idVendor=413c, idProduct=2107
usb 1-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
EXT4-fs (mmcblk0p2): first meta block group too large: 1 (group descriptor block count 1)
List of all partitions:
[...]
 b300 3817472 mmcblk0
  driver: mmcblk
   b301  131072 mmcblk0p1 642536ac-01
   b302 3685376 mmcblk0p2 642536ae-02
 b360 512 mmcblk0rpmb
  (driver?)
 b340 4096 mmcblk0boot1
  (driver?)
 b320 4096 mmcblk0boot0
  (driver?)
No filesystem could mount root, tried:
 ext4
Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(179,2)

After the kernel panic, I'm not able to acces the CM on my host pc (using rpiboot from raspberry-tools), and I get an error:

Error mounting /dev/sdg2 at /media/user/root: Command-line mount -t "ext4" -o "uhelper=udisks2,nodev,nosuid" "/dev/sdg2" "/media/user/root"' exited with non-zero exit status 32: mount: wrong fs type, bad option, bad superblock on /dev/sdg2

Which I'm not able to correct:

$ sudo fsck.ext4 /dev/sdg2
e2fsck 1.42.13 (17-May-2015)
root: clean, 27461/208800 files, 197608/921344 blocks

Any suggestions ? :)

TheSin- commented 7 years ago

I am now seeing this issue and am trying to sort it. It happens after the roots resizes, so something must be happening during the

resize2fs /dev/mmcblk0p2

hopefully I'll have it sorted today, now that it's happening to me it should be easier to diagnose.

TheSin- commented 7 years ago

with that last commit, with my tests it's working can you please confirm when you have time?

mzemb commented 7 years ago

I confirm that the commit 93768db solves the problem for me, thank you. :)