search

openhab / openhabian

openHABian - empowering the smart home, for Raspberry Pi and Debian systems
https://community.openhab.org/t/13379
ISC License
821 stars 251 forks source link

Boot up fails with Kernel panic after upgrade (ua-netinst) #147

Closed ghost closed 5 years ago

ghost commented 7 years ago

Image "openhabianpi-ua-netinst-20170317-gitbd31755f-crc88a5ae46.img" booting on RapberryPi 3 getting error in screenshot on first startup/boot. I have tried on two different SD Cards, but same error. 2017-06-04 00 33 42

the-habu commented 7 years ago

Same problem here. Encountered after updating on rpi3.

ghost commented 7 years ago

Just tried with version 1.1 and getting same error.

ThomDietrich commented 7 years ago

Hey guys, that's a serious issue :-/ Are you both working with the "ua-netinst" image? As stated in the releases section I'd recommended to use the raspbian image from here on out. The next release will not include the ua-netinst build any longer.

I'm on a trip and will be able to do further testing on Tuesday. Please try with the raspbian image and let me know of the results.

Btw. I hope you are normally working via SSH ;)

ghost commented 7 years ago

Hello, always SSH :) Tried with the raspbian image and all is working :)

shoopdawoop commented 7 years ago

Not a fix for the issue, but I had the same issue after an apt-get upgrade of my setup based on openhabian-ua-netinst-20161216-git52cc420.img; I suppose the latest RPi firmware did not play nice with the older 4.4.0-1-rpi2 kernel I was running and I got stuck with a kernel panic after a reboot. Simply updating the boot partition (with a newer kernel) did boot up fine so I could grab a backup, but a lot of stuff was broken (homegear, docker) so I ended up starting over with a fresh install based on openhabianpi-raspbian-20170318-git9767b24-crc9616e8a1.img.xz, which worked fine for me.

johannesbonn commented 7 years ago

+1 same problem for me. @shoopdawoop: Can you please explain your fix with the boot partition? Can you point me to the newer kernel and the steps to boot up the system? I have a serious problem and need openhanded running.

@ThomDietrich: I work with the ua-netinst image. Time for switching I think:-)

Thank you!

Have a nice evening

Johannes

the-habu commented 7 years ago

+1 same problem for me as @shoopdawoop. Also broke my openhabian-ua-netinst-? img. Also started over with a fresh install in my panic... Always working via SSH. Not sure what other options I might have for "working". ;)

shoopdawoop commented 7 years ago

@johannesbonn I put openhabianpi-raspbian-20170318-git9767b24-crc9616e8a1.img on another SD card and then copied the content of the /boot/ partition to the /boot/ partition of the old (broken) SD card, which then boots fine with the new kernel. I have no idea what the implications of this are, and as I said a lot of stuff no longer worked. I just went this way to scp some files from the old cards /root/ over to another machine as this was the most convenient way for me.

LaTell commented 7 years ago

Same Problem for me on a Rpi3. I had a running System, but after installed the Update from openhabian-config i get the kernel Panic at the next Reboot. Now i get also the kernel panic after the first Reboot from a fresh net install. I tried 2 different SD Cards.

johannesbonn commented 7 years ago

@shoopdawoop Thank you!!!!!

ThomDietrich commented 7 years ago

Hey everyone, let me summarize. This issue is of concern to users which are still using a ua-netinst based setup and which recently performed an Update? Which "update" are we talking about? I don't think openHABian applied any changes recently, which would bring around a kernel panic. From what I read by @shoopdawoop this is related to a kernel package upgrade which came via apt or is this related to a rpi-update issued firmware upgrade? The latter is a known issue. From your experience, what should be the recommendation for other ua-netinst users at this point? I want to excuse my reaction time, I'm quite swamped with other tasks this week and will try to tackle the issue in the next couple of days.

shoopdawoop commented 7 years ago

@ThomDietrich: These were the packages that I installed via apt-get upgrade no other updates were offered at the time (I think): libraspberrypi-bin libraspberrypi0 raspberrypi-bootloader-nokernel.

Edit: From what I recall I got the same kernel panic as @MrFrostmaul.

Full transcript: upgrade-2017-06-03.txt

tlrider99 commented 7 years ago

@ThomDietrich I did used the menu point "update OS" in Openhabian and after a reboot the system are running in a kernal panic, after this i tried to write the openhabianpi-raspbian-20170318-git9767b24-crc9616e8a1 to a new sd card with the same result.

LaTell commented 7 years ago

@ThomDietrich I copied an Backup from 14 days ago to my SD Card and my Pi is running without Kernel Panic. When i use the Upgrade function in openhabian-config (Menue 02) my Pi then i get the Kernel Panic after the next Reboot. The login Screen shows 4 apt-get Updates apt updates

The same, when i try a new Netinstallation

I hope you can help us

derM0j0 commented 7 years ago

Hi,

discovered today that i have the same issue... as i power cycled my pi to move it somewhere else. Directly after update it was working fine. Unfortunately i have no backup :-(. Is there a way to rerieve my config files with Windows machine to than reinstall openhab.

Saludos M0j0

the-habu commented 7 years ago

Hi @derM0j0, try one of the tools mentioned here https://www.howtogeek.com/112888/3-ways-to-access-your-linux-partitions-from-windows/. I used Ext2explore to extract the filesystem from the Pi3 SD Card.

kainhofer commented 7 years ago

I have the same issue on my openhabian, which I set up a few months ago.

Any idea how to get the raspberry boot again without starting from scratch, as I spent most of the last few months setting up other stuff in the raspbian, too (mqtt broker, grafana, influxdb with collect also from other servers, several hand-written scripts to read BLE devices, the RFLink, proper nginx reverse proxyies to all services, etc.)?
So, simply reinstalling openhabian and copying over the OpenHAB config files is not an option, as I spent so much time properly setting up the devel environment on the raspberry and setting up all the other stuff...

I can easily mount the sd card on my Linux laptop (both the mmcblk0p1 FAT and the mmcblk0p2 ext partition, and they appear fine (fsck also shows no filesystem corruption, except for the clean unmount of the FAT partition). How can I fix the kernel to boot again?

ThomDietrich commented 7 years ago

@kainhofer please see here:

Could you please try removing the line enable_uart=1 and booting again?

kainhofer commented 7 years ago

@ThomDietrich Thanks, commenting out the line enable_uart=1 from the config.txt on the mmcblk0p1 partition (FAT, so should be doable with Windows machines, too) really makes my openhabian boot again....

LaTell commented 7 years ago

@ThomDietrich My Pi is also Booting after i comment out the enable_uart=1 from the Config.txt

Many Thanks !!! Tell

derM0j0 commented 7 years ago

@ThomDietrich worked for me as well, but keep in mind to add the smsc95xx.macaddr=00:00:00:00:00:00 line as weel as described in https://community.openhab.org/t/unable-to-boot-on-rpi3-with-latest-update/29354/16 to have a static macaddress or you will run into changing IP/MAC issues

ThomDietrich commented 7 years ago

Hey guys, not exactly a fix for the issue but I've added a warning to the openHABian configuration tool. Hopefully it will save a few users from the headache: https://github.com/openhab/openhabian/pull/153 Could one of you please test if the warning is shown?

Edit: Warning redacted and moved below.

zigulle commented 7 years ago

Hello,

after a update I ran into the same problem. Commenting out the line enable_uart=1 fixed the problem, but now I am not able to use the knx-uart-module. Please fix the problem, so I can use the knx-uart-module again.

Greetings, Daniel

Boby71 commented 7 years ago

Stupid question: How do I know if I am using the "ua-netinst based setup"?

I'm running on a RPi2 and I'm missing the enable_uart=1 line in the config.txt file, so nothing to comment out for me. But I only updated the openhabian-config itself, but refrained from starting the "upgrade". How should I proceed?

Thanks, Boby

ThomDietrich commented 7 years ago

@Boby71

I've added a warning to the openHABian configuration tool.

If you do not see a warning there is no problem ;)

Boby71 commented 7 years ago

@ThomDietrich: Okay - I got it. I guess this means, completely new installation...

ThomDietrich commented 7 years ago

@Boby71 no, not sure what you are trying to do but if (1) your system is working as expected and (2) you are not seeing any warning messages when executing sudo openhabian-config, then there is no issue to solve.

To bring your system up to date, just execute Update, then Upgrade. After that you should run all elements from within "Apply improvements" to be up to date. The last thing you might want to do is install the openHAB Log Viewer from within "Optional Components".

Boby71 commented 7 years ago

I got this message after I updated openhabian-config yesterday, but I refrained from hit the upgrade button. Means: I still run on the "old" version because I didn't execute the "upgrade" function. Now I'm unsure if I should trigger the "upgrade" since this message is coming up when starting openhabian-config

ThomDietrich commented 7 years ago

You should not. In this case the best option is indeed to take the chance and switch over to the (overall better) Raspbian Image.

gargamelonly commented 7 years ago

Sorry for lame question. How do I know my image is a "ua-netinst" based image? I assume that if I see the warning message I have it.

  1. I pressed upgrade
  2. Got Kernel panic.
  3. Commented UART in config.txt
  4. System now boots normally. The install I have was the openHABianPi image downloaded from the project home. Do I need a compete re-install with the correct image or is there a way to update and switch to the recommended image?
helmar74 commented 7 years ago

Also sorry for my stupid question. ;-) I get this error after starting the tool: | Attention: It was brought to our attention that the old openHABian │ │ ua-netinst based image has a problem with a lately updated Linux package. │ │ If you upgrade(d) the package 'raspberrypi-bootloader-nokernel' your │ │ Raspberry Pi will run into a Kernel Panic upon reboot! │ │ │ │ Do not Upgrade, do not Reboot! │ │ │ │ A preliminary solution is to not upgrade the system (via the Upgrade menu │ │ entry or 'apt upgrade') or to modify a configuration file. In the long run │ │ we would recommend to switch over to the new openHABian Raspbian based │ │ system image! This error message will keep reapearing even after you fixed │ │ the issue at hand. │ │ Please find all details regarding the issue and the resolution of it at: │ │ https://github.com/openhab/openhabian/issues/147

Is my system bootin, if I only comment out uart in config.txt? I'm also using a zwave usb adapter. Is it working after upgrading the system an rebooting?

Thanks a lot

Regards

helmar

szymic1 commented 7 years ago

What does "we would recommend to switch over to the new openHABian Raspbian based" mean. When I use openhabian 1.3 is it ok? What does 'new' mean ? I've found in release notes that even in openhabian 1.3 there is a warning about this issue "Warn about ua-netinst kernel panic (#153)"

ThomDietrich commented 7 years ago

@szymic1 openHABian started off as an image based on the ua-netinst project. With version 1.2 I've introduced an image based on the standard raspbian lite system. With it's proven benefits we've decided to go with raspbian lite from here on out and discontinued the ua-netinst image.

So to answer your question: Yes you want to start anew with th image file openhabianpi-raspbian-201706161358-git7ff273e-crc00389b9f.img.xz.

NCO3 commented 7 years ago

I see a lot about the Raspberry Pi 3 - is this valid for the Raspi 2 as well?

ThomDietrich commented 7 years ago

Resolution

If you are seeing the error message referring you here, you are probably using the ua-netinst based setup of openHABian on your RPi (all models). If unsure check for the file /boot/config-reinstall.txt which is only available on ua-netinst based systems.

Please do not upgrade the package raspberrypi-bootloader-nokernel! An upgrade of this package will result in a Kernel Panic during the next reboot of the system.

To prevent the upgrade execute the command: 

sudo apt-mark hold raspberrypi-bootloader-nokernel

If you already upgraded the package or tried to reboot without success after upgrading, apply the following workaround to the files on your SD card (also possible while plugged to a Windows PC) in order to be able to (re)boot:

The fix is just a workaround and will disable your serial peripherals.

We recommend to reinstall your system with the Raspbian based openHABian system as soon as possible.

More info:

Best regards, sorry for the trouble and as always: Happy Hacking!

bennybubble commented 6 years ago

dear all,

I'm still baffled on what to do (probably due to my English) :

I have an ua-netinst based system with openHAB 2.0.0-1 up and running, made a lot of modifications/addons to it but system is working fine and stable so far:

############################################################################### ################# openHAB2 ######################################## 7.05.170509 ############################################################################### Ip = 1.2.3.4 Release = Raspbian GNU/Linux 8 (jessie) Kernel = Linux 4.4.0-1-rpi2 Platform = Raspberry Pi 3 Model B Rev 1.2 Uptime = 143 day(s). 20:54:25 CPU Usage = 0 % avg over 4 cpu(s) (4 core(s) x 1 socket(s)) CPU Load = 1m: 0.17, 5m: 0.18, 15m: 0.20 Memory = Free: 0.10GB (11%), Used: 0.84GB (89%), Total: 0.94GB Swap = Free: 0.00GB (0%), Used: 0.00GB (100%), Total: 0.00GB Root = Free: 10.57GB (76%), Used: 3.16GB (24%), Total: 14.49GB Updates = 104 apt-get updates available. Sessions = 2 sessions Processes = 149 running processes of 32768 maximum processes MySQL = Version: 14.14 Distribution: 5.5.54 PHP = Version: 5.6.30, Memory Limit:

          Welcome to            __  _____    ____  _
        ____  ____  ___  ____  / / / /   |  / __ )(_)___ _____
       / __ \/ __ \/ _ \/ __ \/ /_/ / /| | / __  / / __ `/ __ \
      / /_/ / /_/ /  __/ / / / __  / ___ |/ /_/ / / /_/ / / / /
      \____/ .___/\___/_/ /_/_/ /_/_/  |_/_____/_/\__,_/_/ /_/
          /_/
              openHAB 2.0.0-1 (Release Build)

At that time I performed an update and had the panic error, I was able to recover a backup (!) and system was up within one hour but still running the 'original openHABian' version. Since it happened I did not have enough time to look into this but now I have and came across this github openhab issues article:

We recommend to reinstall your system with the Raspbian based openHABian system as soon as possible.

Does this mean I have to make all changes again or is there a better/quicker way? For me it means installing again MQTT/Grafana/INFLUXDB and a lot of software packages.

When openHAB 2.0 was out (based on openRaspbian) I already went through this and hoped never to do this again but real life is different :-)

Any feedback really appreciated!

-ben

ThomDietrich commented 6 years ago

Hello @bennybubble, first to get your setup stable again you should follow the instructions I've given above.

Now you should indeed consider installing a fresh setup. The Raspbian based image brings quite a few improvements with it. Just do a backup of your settings, then install the latest openHABian image, then apply all changes and install additional components via sudo openhabian-config. Things like Mosquitto or Grafana are available there. If something is not working as expected or missing so far: Report an issue! Overall this shouldn't take more than a few hours.

Btw. when talking about "the latest openHABian image": I'll release a new image based on Raspbian Stretch in a few days, you should take the chance! Look out for the official announcement.

bennybubble commented 6 years ago

Hi Thom,

Thanks for your feedback, reading all the info I started a fresh installation based upon the 'latest image', one day ago. I assume I can update this to the 'newer' version using the openhabian-config tool, is this correct?

-ben

ThomDietrich commented 6 years ago

Yes ;)

bennybubble commented 6 years ago

Great, thanks!

buzink commented 6 years ago

The fix is just a workaround and will disable your serial peripherals

@ThomDietrich , what does this mean in the context of the raspberry pi? Does this mean that all hats/shields/daughterboards do not work after holding back raspberrypi-bootloader-nokernel and upgrading on the old openhabian platform, but the rest works?

EliasGabrielsson commented 5 years ago

I think this issue have played out its role as old systems have probably been cycled or reinstalled. Closing this now. Please check out Thomas' workaround solution above.