search

raspberrypi / linux

Kernel source tree for Raspberry Pi-provided kernel builds. Issues unrelated to the linux kernel should be posted on the community forum at https://forums.raspberrypi.com/
Other
11.15k stars 5k forks source link

SSD boot failed with 5.10.5-v8+ #4055

Open Makurisan opened 3 years ago

Makurisan commented 3 years ago

If I actualize to version 5.10.5-v8+ the ssd which is connected to an usb port is unable to boot. The screen is black with an cursor blinking on the upper left corner. I'v done the update with "Hexxeh/rpi-update". The system is arm64. The same problem is with a fresh install and update to 5.10.5-v8. I have no problem with sdcard and updating to 5.10.5-v8+. It is an pi 4 with 4GB. the eeprom is up to date with11.dec.2020

IComplainInComments commented 3 years ago

Was the filesystem f2fs by chance? I have a Pi 3b+ that was just updated to 5.10.5 64bit Raspbian OS, updated through APT, and it corrupted the filesystem to the point where I have to reinstall (As such I'm moving to Arch Linux ARM as its more stable than Raspberry Pi OS 64bit).

Maybe pop the SSD in on another computer and check what Gparted is saying?

Exaga commented 3 years ago

Was the filesystem f2fs by chance? I have a Pi 3b+ that was just updated to 5.10.5 64bit Raspbian OS, updated through APT, and it corrupted the filesystem to the point where I have to reinstall (As such I'm moving to Arch Linux ARM as its more stable than Raspberry Pi OS 64bit).

Same with kernel 5.10.5 on the RPi3, and the RPi2 but using SD card and not SSD. Terminal root partition filesystem corruption after updating and running the system for a short period of time. It initially seemed fine on reboot but after 15-20 minutes is when the problem(s) manifested themselves; dropping the network c/w terminal filesystem corruption. Ergo, I'm against upgrading the RPi (1) lest it goes the same way. RPi4 has been upgraded with the same kernel/boot-firmware and seems stable for now. I'm running Slackware ARM (32bit) OS. I've noticed some quirkiness with the RPi kernel 5.10.x major revision branch but 5.10.2 seems the most stable, thus far. Spent the entire day yesterday trying to investigate and resolve the issues on the RPi3B+ thinking it was first corrupt SD card(s) and then possibly the device itself. Then I tested with my other RPi3B and RPi2 which produced exactly the same results using kernel 5.10.5 and the latest boot-firmware commits. Tested several SD cards on the 2 seperate RPi3's and the RPi2 - all with the same outcome.

I put one of the suspected-corrupted SD cards in the RPi4 via USB card reader to check it out... which is '/dev/sda' in this instance, with the root partition being '/dev/sda3' and should be ext4. RPi4 is currently running kernel 5.10.5 with the latest boot-firmware:

root@torq:/# parted -l
Model: Generic STORAGE DEVICE (scsi)
Disk /dev/sda: 32.0GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:

Number  Start   End     Size    Type     File system     Flags
 1      16.4kB  160MB   160MB   primary  fat32           boot, lba
 2      160MB   2308MB  2147MB  primary  linux-swap(v1)
 3      2308MB  32.0GB  29.7GB  primary

Model: SD SD64G (sd/mmc)
Disk /dev/mmcblk0: 62.3GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:

Number  Start   End     Size    Type     File system     Flags
 1      16.4kB  160MB   160MB   primary  fat32           boot, lba
 2      160MB   1234MB  1074MB  primary  linux-swap(v1)
 3      1234MB  62.3GB  61.0GB  primary  ext4
root@torq:/# blkid /dev/sda3
/dev/sda3: PARTUUID="11e6d640-03"
root@torq:/# fsck /dev/sda3
fsck from util-linux 2.36.1
e2fsck 1.45.6 (20-Mar-2020)
ext2fs_open2: Bad magic number in super-block
/sbin/e2fsck: Superblock invalid, trying backup blocks...
Journal version not supported by this e2fsck.
root@torq:/# fsck /dev/sda3
fsck from util-linux 2.36.1
e2fsck 1.45.6 (20-Mar-2020)
ext2fs_open2: Bad magic number in super-block
/sbin/e2fsck: Superblock invalid, trying backup blocks...
Journal version not supported by this e2fsck.
Abort? yes

/dev/sda3: ***** FILE SYSTEM WAS MODIFIED *****

/dev/sda3: ********** WARNING: Filesystem still has errors **********
root@torq:/# lsblk -f
NAME        FSTYPE FSVER LABEL    UUID                                 FSAVAIL FSUSE% MOUNTPOINT
sda
├─sda1      vfat   FAT32 rpi2boot 6029-5EED
├─sda2      swap   1              d45b2d28-3654-4315-a25f-851910344f14
└─sda3
mmcblk0
├─mmcblk0p1 vfat   FAT32          17A2-107A                             105.5M    30% /boot
├─mmcblk0p2 swap   1              2b7af18d-f0df-42c0-b291-c20c9a7a7624                [SWAP]
└─mmcblk0p3 ext4   1.0            6d475152-d9f4-4ad8-ba63-20e2938796ff   41.2G    21% /
root@torq:/# fdisk /dev/sda3

Welcome to fdisk (util-linux 2.36.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table.
Created a new DOS disklabel with disk identifier 0xcc0ef6bb.

Command (m for help):
iucoen commented 3 years ago

I think this was caused by a kernel bug. I rebuilt the kernel with latest changes today and it seem to have been fixed.

r3dnck commented 3 years ago

Same issue with kernel bcf09c55a208f82aad1b20e55bdcb11dc46e048e

iucoen commented 3 years ago

Same issue with kernel bcf09c5

If a bad kernel corrupted the file system, it stays forever corrupted. I had to restore the SD card from an image. I did this multiple times, and each time I updated to the bad kernel it ends up corrupting my file system.

I think this is the change that fixed the issue for me: a896a00ebffd0b016627f65ff236b660e66e782b

r3dnck commented 3 years ago

If a bad kernel corrupted the file system, it stays forever corrupted.

I fixed the filesystem with fsck and rolled back the kernel to 5.10.4. Everything works fine so far.

Makurisan commented 3 years ago

I have done several things trying to solve it but nothing helps. I reinstalled the image on the ssd and now work with 5.10.4. I try it again if I have a second ssd.

Exaga commented 3 years ago

I have done several things trying to solve it but nothing helps. I reinstalled the image on the ssd and now work with 5.10.4. I try it again if I have a second ssd.

Same. I spent many hours trying to fathom why the root filesystem was being corrupted without success. I too switched back to 5.10.4 which seems to run with some degree of reliability and stability. Hopefully with the release of 5.10.6 the problem has now been resolved. Fingers-crossed.

pelwell commented 3 years ago

Does this comment help: https://github.com/raspberrypi/rpi-eeprom/issues/266#issuecomment-756005465

timg236 commented 3 years ago

There was a new bootloader release yesterday which should be in APT soon, however, the XHCI and PCIe are reset when Linux is started so if it's unlikely that the bootloader update will have much impact after this state of the boot.