search

hardkernel / linux

Linux kernel source tree
Other
426 stars 407 forks source link

4.14.19 xu4: kernel does not boot with CONFIG_CGROUP_PIDS set #337

Open paralin opened 6 years ago

paralin commented 6 years ago

Boot script:

ODROIDXU-UBOOT-CONFIG

# U-Boot Parameters
setenv initrd_high "0xffffffff"
setenv fdt_high "0xffffffff"
# setenv silent 1
setenv condev "console=ttySAC2,115200n8"

# setenv dtb_addr_r "0x44000000"
#setenv kernel_addr_r "0x40008000"
#setenv dtb_addr_r "0x467BA800"
#setenv initramfs_addr_r "0x46834800"

#setenv dtb_addr_r "0x42000000"
#setenv kernel_addr_r "0x40008000"
#setenv initramfs_addr_r "0x44000000"
#setenv dtb_addr_r "0x42000000"

setenv kernel_addr_r "0x40008000"
setenv initramfs_addr_r "0x46000000"
setenv dtb_addr_r "0x44000000"

# Mac address configuration
# setenv macaddr "00:1e:06:61:7a:39"
# setenv macaddrconfig "smsc95xx.macaddr=${macaddr}"

#---------------------------------
# Boots from memory.
# --------------------------------
setenv bootmem "console=tty1 ${condev} root=/dev/initrd ro ramdisk_size=100000 fsck.repair=yes net.ifnames=0 no_console_suspend"

# --- Screen Configuration for HDMI --- # 
# ---------------------------------------
# Uncomment only ONE line! Leave all commented for automatic selection.
# Uncomment only the setenv line!
# ---------------------------------------
# ODROID-VU forced resolution
# setenv videoconfig "video=HDMI-A-1:1280x800@60"
# -----------------------------------------------
# ODROID-VU forced EDID
# setenv videoconfig "drm_kms_helper.edid_firmware=edid/1280x800.bin"
# -----------------------------------------------
# 1920x1200 60hz without monitor data using generic information
# setenv videoconfig "drm_kms_helper.edid_firmware=edid/1920x1200_60hz.bin"
# -----------------------------------------------
# 1920x1200 30hz without monitor data using generic information
# setenv videoconfig "drm_kms_helper.edid_firmware=edid/1920x1200_30hz.bin"
# -----------------------------------------------
# 1920x1080 (1080P) with monitor provided EDID information. (1080p-edid)
# setenv videoconfig "video=HDMI-A-1:1920x1080@60"
# -----------------------------------------------
# 1920x1080 (1080P) without monitor data using generic information (1080p-noedid)
# setenv videoconfig "drm_kms_helper.edid_firmware=edid/1920x1080.bin"
# -----------------------------------------------
# 1920x1080 50hz (1080P) with monitor provided EDID information. (1080p 50hz-edid)
# setenv videoconfig "video=HDMI-A-1:1920x1080@50"
# -----------------------------------------------
# 1920x1080 50hz (1080P) without monitor data using generic information (1080p 50hz-noedid)
# setenv videoconfig "drm_kms_helper.edid_firmware=edid/1920x1080_50hz.bin"
# -----------------------------------------------
# 1680x1050 without monitor data using generic information 
# setenv videoconfig "drm_kms_helper.edid_firmware=edid/1680x1050.bin"
# -----------------------------------------------
# 1600x1200 without monitor data using generic information 
# setenv videoconfig "drm_kms_helper.edid_firmware=edid/1600x1200.bin"
# -----------------------------------------------
# 1600x900 without monitor data using generic information 
# setenv videoconfig "drm_kms_helper.edid_firmware=edid/1600x900.bin"
# -----------------------------------------------
# 1440x900 with monitor provided EDID information.
# setenv videoconfig "video=HDMI-A-1:1440x900@60"
# -----------------------------------------------
# 1440x900 without monitor data using generic information 
# setenv videoconfig "drm_kms_helper.edid_firmware=edid/1440x900.bin"
# -----------------------------------------------
# 1366x768 without monitor data using generic information 
# setenv videoconfig "drm_kms_helper.edid_firmware=edid/1366x768.bin"
# -----------------------------------------------
# 1360x768 without monitor data using generic information 
# setenv videoconfig "drm_kms_helper.edid_firmware=edid/1360x768.bin"
# -----------------------------------------------
# 1280x1024 without monitor data using generic information
# setenv videoconfig "drm_kms_helper.edid_firmware=edid/1280x1024.bin"
# -----------------------------------------------
# 1280x720 (720P) with monitor provided EDID information. (720p-edid)
# setenv videoconfig "video=HDMI-A-1:1280x720@60"
# -----------------------------------------------
# 1280x720 (720P) without monitor data using generic information (720p-noedid)
# setenv videoconfig "drm_kms_helper.edid_firmware=edid/1280x720.bin"
# -----------------------------------------------
# 1024x768 without monitor data using generic information
# setenv videoconfig "drm_kms_helper.edid_firmware=edid/1024x768.bin"
# -----------------------------------------------
# 1024x600 without monitor data using generic information (ODROID VU7+)
# setenv videoconfig "drm_kms_helper.edid_firmware=edid/1024x600.bin"
# -----------------------------------------------
# 800x600 without monitor data using generic information 
# setenv videoconfig "drm_kms_helper.edid_firmware=edid/800x600.bin"
# -----------------------------------------------
# 800x480 without monitor data using generic information (ODROID 7")
# setenv videoconfig "drm_kms_helper.edid_firmware=edid/800x480.bin"
# -----------------------------------------------
# 720x576 without monitor data using generic information 
# setenv videoconfig "drm_kms_helper.edid_firmware=edid/720x576.bin"
# -----------------------------------------------
# 720x480 without monitor data using generic information 
# setenv videoconfig "drm_kms_helper.edid_firmware=edid/720x480.bin"
# -----------------------------------------------
# 640x480 without monitor data using generic information
# setenv videoconfig "drm_kms_helper.edid_firmware=edid/640x480.bin"
# -----------------------------------------------
# 480x800 without monitor data using generic information
# setenv videoconfig "drm_kms_helper.edid_firmware=edid/480x800.bin"

# --- HDMI / DVI Mode Selection ---
# ------------------------------------------
# - HDMI Mode
# setenv vout "hdmi"
# - DVI Mode (disables sound over HDMI as per DVI compat)
# setenv vout "dvi"

# --- HDMI CEC Configuration ---
# ------------------------------------------
# setenv cecenable "false" # false or true
# set to true to enable HDMI CEC

# Enable/Disable ODROID-VU7 Touchsreen
setenv disable_vu7 "true" # false

# CPU Governor Selection
# Available governos: conservative, userspace, powersave, ondemand, performance, schedutil
setenv governor "performance"

# External watchdog board enable
setenv external_watchdog "false"
# debounce time set to 3 ~ 10 sec, default 3 sec
setenv external_watchdog_debounce "3"

#------------------------------------------------------------------------------
#
# HDMI Hot Plug detection
#
#------------------------------------------------------------------------------
#
# Forces the HDMI subsystem to ignore the check if the cable is connected or 
# not.
# false : disable the detection and force it as connected.
# true : let cable, board and monitor decide the connection status.
# 
# default: true
# 
#------------------------------------------------------------------------------
# setenv HPD "true"

# setenv hdmi_phy_control "HPD=${HPD} vout=${vout}"

# Load kernel, initrd and dtb in that sequence
fatload mmc 0:1 ${kernel_addr_r} zImage
fatload mmc 0:1 ${initramfs_addr_r} rootfs.cpio.uboot
if test "${board_name}" = "xu4"; then fatload mmc 0:1 ${dtb_addr_r} exynos5422-odroidxu4.dtb; setenv fdtloaded "true"; fi
if test "${board_name}" = "xu3"; then fatload mmc 0:1 ${dtb_addr_r} exynos5422-odroidxu3.dtb; setenv fdtloaded "true"; fi
if test "${board_name}" = "xu3l"; then fatload mmc 0:1 ${dtb_addr_r} exynos5422-odroidxu3-lite.dtb; setenv fdtloaded "true"; fi
#failsafe
if test "${fdtloaded}" != "true"; then fatload mmc 0:1 ${dtb_addr_r} exynos5422-odroidxu4.dtb; fi

# 0x42000000 = ${dtb_addr_r}
# fdt addr 0x42000000
fdt addr ${dtb_addr_r}

# if test "${cecenable}" = "false"; then fdt rm /cec@101B0000; fi
if test "${disable_vu7}" = "false"; then setenv hid_quirks "usbhid.quirks=0x0eef:0x0005:0x0004"; fi
if test "${external_watchdog}" = "true"; then setenv external_watchdog "external_watchdog=${external_watchdog} external_watchdog_debounce=${external_watchdog_debounce}"; fi

# final boot args
setenv bootargs "${bootmem} ${videoconfig} ${hdmi_phy_control} ${hud_quirks} ${macaddrconfig} ${external_watchdog} governor=${governor}"

# Boot the board
bootz ${kernel_addr_r} ${initramfs_addr_r} ${dtb_addr_r}

This boot script allows for 67Mb of kernel, while the actual kernel loaded under 4.9:

reading zImage
4948704 bytes read in 334 ms (14.1 MiB/s)

And under 4.14, not much more than 5Mb, I concluded then that overlap cannot be the issue here.

So, I tried building the following kernel versions, making this change:

-BR2_LINUX_KERNEL_CUSTOM_TARBALL_LOCATION="https://github.com/hardkernel/linux/archive/1ebaaab89354d604066acb505f6574d0b926b3ea/linux-xu4-4.14.18-r1.tar.gz"
-BR2_LINUX_KERNEL_VERSION="xu4-4.14.18-r1"
-BR2_PACKAGE_HOST_LINUX_HEADERS_CUSTOM_4_14=y
+BR2_LINUX_KERNEL_CUSTOM_TARBALL_LOCATION="https://github.com/hardkernel/linux/archive/ce6a7ef660cca271a04e23315c131b544548029b/linux-xu4-4.9.52-r1.tar.gz"
+BR2_LINUX_KERNEL_VERSION="xu4-4.9.52"
+BR2_PACKAGE_HOST_LINUX_HEADERS_CUSTOM_4_9=y

Compiling the kernel then with this change, the kernel boots. However, with kernel 4.14.18 at the latest commit (hash in the previous block of text):

U-Boot 2017.05 (Feb 16 2018 - 02:00:37 -0500) for ODROID-XU4

CPU:   Exynos5422 @ 800 MHz
Model: Odroid XU4 based on EXYNOS5422
Board: Odroid XU4 based on EXYNOS5422
Type:  unknown
DRAM:  2 GiB
MMC:   EXYNOS DWMMC: 0, EXYNOS DWMMC: 1
MMC Device 0 ( SD ): 59.5 GiB
Card did not respond to voltage select!
mmc_init: -95, time 11
Warning - bad CRC, using default environment

In:    serial
Out:   serial
Err:   serial
Net:   No ethernet found.
Press quickly 'Enter' twice to stop autoboot:  0 
reading boot.ini
8293 bytes read in 5 ms (1.6 MiB/s)
cfgload: applying boot.ini...
cfgload: setenv initrd_high "0xffffffff"
cfgload: setenv fdt_high "0xffffffff"
cfgload: setenv condev "console=ttySAC2,115200n8"
cfgload: setenv kernel_addr_r "0x40008000"
cfgload: setenv initramfs_addr_r "0x46000000"
cfgload: setenv dtb_addr_r "0x44000000"
cfgload: setenv bootmem "console=tty1 ${condev} root=/dev/initrd ro ramdisk_size=100000 fsck.repair=yes ne
t.ifnames=0 no_console_suspend"                                                                          
cfgload: setenv disable_vu7 "true" # false
cfgload: setenv governor "performance"
cfgload: setenv external_watchdog "false"
cfgload: setenv external_watchdog_debounce "3"
cfgload: fatload mmc 0:1 ${kernel_addr_r} zImage
reading zImage
5787624 bytes read in 390 ms (14.2 MiB/s)
cfgload: fatload mmc 0:1 ${initramfs_addr_r} rootfs.cpio.uboot
reading rootfs.cpio.uboot
58734570 bytes read in 3886 ms (14.4 MiB/s)
cfgload: if test "${board_name}" = "xu4"; then fatload mmc 0:1 ${dtb_addr_r} exynos5422-odroidxu4.dtb; set
env fdtloaded "true"; fi                                                                                 
reading exynos5422-odroidxu4.dtb
63616 bytes read in 9 ms (6.7 MiB/s)
cfgload: if test "${board_name}" = "xu3"; then fatload mmc 0:1 ${dtb_addr_r} exynos5422-odroidxu3.dtb; set
env fdtloaded "true"; fi                                                                                 
cfgload: if test "${board_name}" = "xu3l"; then fatload mmc 0:1 ${dtb_addr_r} exynos5422-odroidxu3-lite.dt
b; setenv fdtloaded "true"; fi                                                                           
cfgload: if test "${fdtloaded}" != "true"; then fatload mmc 0:1 ${dtb_addr_r} exynos5422-odroidxu4.dtb; fi
cfgload: fdt addr ${dtb_addr_r}
cfgload: if test "${disable_vu7}" = "false"; then setenv hid_quirks "usbhid.quirks=0x0eef:0x0005:0x0004"; 
fi                                                                                                       
cfgload: if test "${external_watchdog}" = "true"; then setenv external_watchdog "external_watchdog=${exter
nal_watchdog} external_watchdog_debounce=${external_watchdog_debounce}"; fi                              
cfgload: setenv bootargs "${bootmem} ${videoconfig} ${hdmi_phy_control} ${hud_quirks} ${macaddrconfig} ${e
xternal_watchdog} governor=${governor}"                                                                  
cfgload: bootz ${kernel_addr_r} ${initramfs_addr_r} ${dtb_addr_r}
Kernel image @ 0x40008000 [ 0x000000 - 0x584fe8 ]
## Loading init Ramdisk from Legacy Image at 46000000 ...
   Image Name:   
   Image Type:   ARM Linux RAMDisk Image (uncompressed)
   Data Size:    58734506 Bytes = 56 MiB
   Load Address: 00000000
   Entry Point:  00000000
   Verifying Checksum ... OK
## Flattened Device Tree blob at 44000000
   Booting using the fdt blob at 0x44000000
   Using Device Tree in place at 44000000, end 4401287f

Starting kernel ...

[hang]

How should I proceed with debugging this?

Cross-compiling with GCC 7

paralin commented 6 years ago

Additionally, building with HEAD of the 4.9 tree also works:

-BR2_LINUX_KERNEL_CUSTOM_TARBALL_LOCATION="https://github.com/hardkernel/linux/archive/1ebaaab89354d604066acb505f6574d0b926b3ea/linux-xu4-4.14.18-r1.tar.gz"
-BR2_PACKAGE_HOST_LINUX_HEADERS_CUSTOM_4_14=y

+BR2_LINUX_KERNEL_CUSTOM_TARBALL_LOCATION="https://github.com/hardkernel/linux/archive/be592282a08a2493692448e365e46b52ac715b3f/linux-xu4-4.9.61-r2.tar.gz"
+BR2_PACKAGE_HOST_LINUX_HEADERS_CUSTOM_4_9=y

I'm trying now a build variant with latest 4.14.x with GCC 6 rather than 7, to see if that improves things.

paralin commented 6 years ago

Here are the combinations I've tried so far:

Seems the toolchain may not be the issue.

paralin commented 6 years ago

cc @mdrjr @moonlinux

Dmole commented 6 years ago

If you have not already you might want to post in https://forum.odroid.com/viewforum.php?f=92 .

paralin commented 6 years ago

@Dmole Thanks! Will do now.

paralin commented 6 years ago

Still master and 4.14.20-108 does not boot:

U-Boot 2017.05 (Feb 23 2018 - 07:49:34 -0500) for ODROID-XU4

CPU:   Exynos5422 @ 800 MHz
Model: Odroid XU4 based on EXYNOS5422
Board: Odroid XU4 based on EXYNOS5422
Type:  unknown
DRAM:  2 GiB
MMC:   EXYNOS DWMMC: 0, EXYNOS DWMMC: 1
MMC Device 0 ( SD ): 59.5 GiB
Card did not respond to voltage select!
mmc_init: -95, time 11
*** Warning - bad CRC, using default environment

In:    serial
Out:   serial
Err:   serial
Net:   No ethernet found.
Press quickly 'Enter' twice to stop autoboot:  0 
reading boot.ini
8309 bytes read in 16 ms (506.8 KiB/s)
cfgload: applying boot.ini...
cfgload: setenv initrd_high "0xffffffff"
cfgload: setenv fdt_high "0xffffffff"
cfgload: setenv condev "console=ttySAC2,115200n8"
cfgload: setenv verify 0
cfgload: setenv kernel_addr_r "0x40008000"
cfgload: setenv initramfs_addr_r "0x46000000"
cfgload: setenv dtb_addr_r "0x44000000"
cfgload: setenv bootmem "console=tty1 ${condev} root=/dev/initrd ro ramdisk_size=100000 fsck.repair=yes ne
t.ifnames=0 no_console_suspend"                                                                          
cfgload: setenv disable_vu7 "true" # false
cfgload: setenv governor "performance"
cfgload: setenv external_watchdog "false"
cfgload: setenv external_watchdog_debounce "3"
cfgload: fatload mmc 0:1 ${kernel_addr_r} zImage
reading zImage
5733520 bytes read in 392 ms (13.9 MiB/s)
cfgload: fatload mmc 0:1 ${initramfs_addr_r} rootfs.cpio.uboot
reading rootfs.cpio.uboot
59395574 bytes read in 3900 ms (14.5 MiB/s)
cfgload: if test "${board_name}" = "xu4"; then fatload mmc 0:1 ${dtb_addr_r} exynos5422-odroidxu4.dtb; set
env fdtloaded "true"; fi                                                                                 
reading exynos5422-odroidxu4.dtb
63616 bytes read in 10 ms (6.1 MiB/s)
cfgload: if test "${board_name}" = "xu3"; then fatload mmc 0:1 ${dtb_addr_r} exynos5422-odroidxu3.dtb; set
env fdtloaded "true"; fi                                                                                 
cfgload: if test "${board_name}" = "xu3l"; then fatload mmc 0:1 ${dtb_addr_r} exynos5422-odroidxu3-lite.dt
b; setenv fdtloaded "true"; fi                                                                           
cfgload: if test "${fdtloaded}" != "true"; then fatload mmc 0:1 ${dtb_addr_r} exynos5422-odroidxu4.dtb; fi
cfgload: fdt addr ${dtb_addr_r}
cfgload: if test "${disable_vu7}" = "false"; then setenv hid_quirks "usbhid.quirks=0x0eef:0x0005:0x0004"; 
fi                                                                                                       
cfgload: if test "${external_watchdog}" = "true"; then setenv external_watchdog "external_watchdog=${exter
nal_watchdog} external_watchdog_debounce=${external_watchdog_debounce}"; fi                              
cfgload: setenv bootargs "${bootmem} ${videoconfig} ${hdmi_phy_control} ${hud_quirks} ${macaddrconfig} ${e
xternal_watchdog} governor=${governor}"                                                                  
cfgload: bootz ${kernel_addr_r} ${initramfs_addr_r} ${dtb_addr_r}
Kernel image @ 0x40008000 [ 0x000000 - 0x577c90 ]
## Loading init Ramdisk from Legacy Image at 46000000 ...
   Image Name:   
   Image Type:   ARM Linux RAMDisk Image (uncompressed)
   Data Size:    59395510 Bytes = 56.6 MiB
   Load Address: 00000000
   Entry Point:  00000000
## Flattened Device Tree blob at 44000000
   Booting using the fdt blob at 0x44000000
   Using Device Tree in place at 44000000, end 4401287f

Starting kernel ...

[dead]

Swapping in the zImage for 4.9.x latest works fine. I have also tried disabling some things to make the kernel smaller, but that has not worked either.

mdrjr commented 6 years ago

Can you try cross-compiling with GCC 5? That's what I've been using and the kernel has been booting just fine.

yrzr commented 6 years ago

Same issue with kernel 4.14.x compiled by GCC 6.x

igorpecovnik commented 6 years ago

Try this patch: https://github.com/armbian/build/blob/development/patch/kernel/odroidxu4-next/not-booting-bugfix.patch

rbray89 commented 5 years ago

@paralin Were you ever able to solve this?

paralin commented 5 years ago

@rbray89 no I was not. The link above with the patch also seems dead :(

igorpecovnik commented 5 years ago

Here is an updated link: https://github.com/armbian/build/blob/master/patch/kernel/odroidxu4-default/not-booting-bugfix.patch

We also provide 4.19.y kernel [next branch] which seems to work good ... but need more testings. https://github.com/armbian/build

rbray89 commented 5 years ago

@paralin So I'm working on Porting HassOS to the XU4 (based on buildroot) and found that my board not booting is due to an additional config option I'm adding as a fragment to the odroidxu4 defconfig. I'm narrowing down the option now. I'll update once I find the Config option causing trouble.

igorpecovnik commented 5 years ago

If you want to save some time, our working configs: https://github.com/armbian/build/tree/master/config/kernel

rbray89 commented 5 years ago

@paralin: So I think your suspicion of space being the issue is correct. I went through my config fragment to disable features until it would boot, then I started to add back features while disabling others. Sure enough, it isn't the feature that is causing the issue, rather the size of the zImage. It breaks somewhere in the range of 6879760-6802648 bytes (note that this is the compressed image size, so I'm unsure of the uncompressed size).

Dmole commented 5 years ago

I would expect increasing your fatload address space would solve your issue;

setenv kernel_addr_r "0x40008000" setenv initramfs_addr_r "0x46000000" setenv dtb_addr_r "0x44000000"

or if you don't want to mess around with uboot you can disable all "_DEBUG" in your conf.

rbray89 commented 5 years ago

I was running with the exynos uboot defaults:

 fdt_addr_r=0x43000000
 kernel_addr_r=0x42000000

Tried using:

 kernel_addr_r=0x41000000

but it didn't help

paralin commented 5 years ago

I previously tried moving the images around to various aligned addresses and was never able to make it work. Note that over a serial line uboot prints the size of the loaded compressed zimage into memory. That size never overran the dtb or the uinitrd.

Maybe when the kernel extracts itself it overwrites something important?

@rbray89 I appreciate the armbian configs. Have tried the fix boot .patch but it doesn't fix the issue sadly. It's more likely that armbian selecting less options and having a smaller zimage is the fix.

rbray89 commented 5 years ago

@paralin So it seems that after extensive testing, it is NOT a size issue for me, it just happens to correlate with my config options. There are two config options that I can't enable/use if I want the system to boot. For me, those two options are: CONFIG_CGROUP_PIDS=y and apparmor if I also set it as the default security through the config or kernel params: CONFIG_SECURITY_APPARMOR=y

Still looking at the root cause, more interested in the apparmor issue personally.

Dmole commented 5 years ago

I think the Big.Little CPU scheduler was not implemented upstream until 4.20 so it may be that the hack from Samsung that was ported forward by Odroid is breaking those 2 features.

paralin commented 5 years ago

@rbray89 Okay, I have made a branch with a config file in place to disable that CGROUP_PIDS option:

https://github.com/paralin/skiffos/tree/odroid-4.14

I haven't had a chance to test yet. By the way, have you considered building HassOS on top of something like Skiff? The general idea is to build a modular configuration package manager so that we can "overlay" hassos on top of a stable base XU4 config, etc. You could also centralize issue reporting related to things in Buildroot / the underlying system rather than Hass itself.

paralin commented 5 years ago

@rbray89 Additionally your hassos Docker build files have fallen out of sync with upstream, I'd recommend backporting my patch series which splits docker-engine and docker-cli

rbray89 commented 5 years ago

@paralin HassOS was originally Resin OS based (similar idea to Skiff) but as the project grew and new requirements/features were added (and the fork continued to grow more distant), the decision was made to create an independent supervisor running on a lightweight linux install. I started development after this change, so I'm not privy to all the minutia; I'm just focused on getting the hardkernel boards working with HassOS :)

paralin commented 5 years ago

@rbray89 noted, however, skiff is primarily a configuration package manager for Buildroot, not a full mandated solution like resinos. Essentially what you would be doing is instead of using a Buildroot submodule, you would use a skiff submodule. You could then write configuration packages for each board and separate out the hypervisor stuff into its own config. Fixes to specific boards then would go to the skiff repo.

paralin commented 5 years ago

Can confirm, boots now after I set CONFIG_CGROUP_PIDS=n. Thanks @rbray89

cc @hardkernel @mdrjr