Bootloop when updating to latest HA OS 11.4 on Raspberry Pi 4

[dyarfaradj]

Describe the issue you are experiencing

Yesterday when I updated my Raspberry pi 4 8GB with SSD to latest HA OS 11.4 from 11.3 it resulted in a bootloop. I've tried to restart the Rbpi4 multiple times, I've tried using different USB port for the SSD. I don't have any microHDMI adapter so I can't connect to it.

It's a 256GB SSD and I barely used any space on it, so can't be due to no space left..

What can I do to save the homeassistant configs/files? How can I retrieve a backup from the SSD? Can I read it somehow through my PC?

Is there anything else I can do?

What operating system image do you use?

generic-x86-64 (Generic UEFI capable x86-64 systems)

What version of Home Assistant Operating System is installed?

11.4

Did you upgrade the Operating System.

Yes

Steps to reproduce the issue

1. Update from 11.3 --> 11,4
2. bootloop

3. ...

Anything in the Supervisor logs that might be useful for us?

no logs

Anything in the Host logs that might be useful for us?

bo logs

System information

Raspberry Pi 4 8GB (SSD)

Additional information

No response

[sairon]

The update from 11.3 to 11.4 shouldn't have any effect on Raspberry Pi - the only change reflected in the OS images is #3043 adding a bunch of firmware files, but that hardly makes a difference. What makes you think the device is in bootloop if you have no access to the output? Are you using the SSD as a data disk or as the main system disk?

[dyarfaradj]

@sairon I saw the OS update in HA dashboard and just choose to update it like I've always done. I'm using the SSD as main system disk. I know it's a boot loop because of the lights of the ethernetport as well as my SSD. Same pattern and then starts all over again. And it doesn't even connect to my router so I know for a fact its not booted properly.

[foltz61]

Same with my Pi 5 and USB SSD. It also didn't create a backup automatically like usual for some reason. Luckily I have a recent sdcard install. I booted the pi with the SD and SSD and it's using the data from the SSD snd I'm able to create a backup to get back to pre update setup.

[dyarfaradj]

I managed to connect the SSD to my PC and download the backup files, 4 days old but it's something...

What should I do now? Reflash/reformat the SSD with Homeassistant and then once installed, restore from the backup? Will I get back everything? Like automations, zigbee2mqtt devices etc?

I cba to re-pair everything once again

[leonardpitzu]

same for me on a pi4 and pi5... lost quite some work I've done today, before rebooting...

[foltz61]

I managed to connect the SSD to my PC and download the backup files, 4 days old but it's something...

What should I do now? Reflash/reformat the SSD with Homeassistant and then once installed, restore from the backup? Will I get back everything? Like automations, zigbee2mqtt devices etc?

I cba to re-pair everything once again

You'll get everything back as far back as your most recent full backup and any addon backups done after that. If you can see your config folder on the pc I'd copy that also. That way you can manuy restore scripts and automations if necessary. I'm not qualified enough to tell you if you should reinstall and restore yet. Someone may know of a way to get you going without doing that.

[dyarfaradj]

I managed to connect the SSD to my PC and download the backup files, 4 days old but it's something... What should I do now? Reflash/reformat the SSD with Homeassistant and then once installed, restore from the backup? Will I get back everything? Like automations, zigbee2mqtt devices etc? I cba to re-pair everything once again

You'll get everything back as far back as your most recent full backup and any incremental stuff done. If you can see your config folder on the pc I'd copy that also. That way you can manuy restore scripts and automations if necessary. I'm not qualified enough to tell you if you should reinstall and restore yet. Someone may know of a way to get you going without doing that.

Alright, thank you. Yeah I've copied everything, including the homeassistant folder with the config, automations scripts etc just in case..

I'll wait few hours and see if anyone knows how to solve this without having to reinstall and restore.

[foltz61]

Here's my homeassistant.log.1 from a failed boot Not sure if it will help because after it didn't boot correctly I was frequently unplugging and plugging my pi back in. HassLog.txt

edit: The log may be from right before I did the update and not a failed boot.

[EnzoD86]

Hi, I also had the same problem on my Raspberry PI 5 with installation on SD memory (purchased less than a month ago). Since I installed hassio on Pi 5, every time I updated HA OS I had this bootloop problem. Initially I thought it was because the release was still a candidate. In these weeks (since HA OS for PI5 was released) I will have restored my SD about 4 times (practically with every new release). The latest one was yesterday (HA OS 11.4)... although in this case I didn't expect it to break, as it was updated from a stable release 11.3 to another stable release 11.4. I can only provide a screenshot of the first boot loop that happened to me when I upgraded from 11.3.rc1 to 11.3.rc2 I hope it can help you understand the problem. Thank you

[foltz61]

I setup 11.3 on my pi 5, using sd instead of ssd, and restored my backup. Then installed 11.4 and everything is working fine. I will do the same to SSD tomorrow and post the results.

[sairon]

In the case of RPi 5 that @EnzoD86 provided the screenshot from, it is most likely broken cmdline.txt, or something in the boot partition - I created a separate issue for that. For the other issues, if the device is stuck in a boot loop, at least a photo of the console output like the above is needed to move forward.

[C0MTRAYA]

Same here, 11.2 to 11.4 stuck in bootloop or just dont boot anymore. Used my sd clone that ive done last week and it work again, ill do another clone and roll the yerterdays backup on it after work to get everything back.

Using rpi4 4gb.

[foltz61]

I redid my SSD and installed 11.4 on top of a fresh 11.3 flash. Everything went fine. I restored my backup but my zwave stuff wasn't working. I rebooted and was stuck in the bootloop again. I then went back to my working sd from last night. I can only assume that the only reason the sd worked last night is because I didn't reboot after the upgrade to 11.4. I ordered a micro hdmi cable so I can be more helpful in the future.

[erkr]

The update from 11.3 to 11.4 shouldn't have any effect on Raspberry Pi - the only change reflected in the OS images is #3043 adding a bunch of firmware files...

And Remove rootfstype for Raspberry Pi 5 (#3037)

Could that be related?

[dyarfaradj]

The update from 11.3 to 11.4 shouldn't have any effect on Raspberry Pi - the only change reflected in the OS images is #3043 adding a bunch of firmware files...

And Remove rootfstype for Raspberry Pi 5 (#3037)

Could that be related?

Probably, can't debug though since I don't have any micro HDMI cable atm

[dyarfaradj]

I just tried to flash a fresh copy of 11.4 to my SSD and boot it with the RBPi4 wihtout any success. Since I don't have micro HDMI cable I can't give any logs.

But as soon as I flashed 11.3 to the same SSD and booted the RBPi4 it worked, got access to http://homeassistant.local:8123/

So something is indeed wrong with 11.4.

[erkr]

@sairon @agners Thanks for all your efforts! Just curious if you already have a fix for this issue in the pipeline. I'm stuck on 11.1

[foltz61]

I now have the proper hdmi cable to get a screenshot of the error on my pi 5. I'm guessing the error is the same as was found for @EnzoD86. If you still want a screenshot let me know and I'll do the update again.

[dwgtx]

I'm running HAOS on an Intel NUC. Everything was running well (11.3) until the upgrade to 11.4, after which I was unable to access the homeassistant.local port. Viewing the NUC locally revealed a cifs_mount failed w/return code = -101 error, which means the OS starts before the network connection is established. Tried several 11.4 reinstalls to no avail - same issue after each reboot. Note that HAOS 11.4 appears to boot correctly but without network access. I did some research and saw suggestions about disabling IPV6, or modifying config files to add a delay to wait for an established network, but I've never had to worry about that prior to 11.4, and don't need want to worry about it now. I'll wait for the HA team 11.4.1 release which fixes their mistake. I reinstalled 11.3 and restored my last daily backup. It's nice to be back in Home Automation Heaven...

[alanburchill]

Also have this issue.... Power off or restart will go into boot loop. Running Raspberry Pi 5 with 11.4

[alanburchill]

haos_rpi5-64-11.5.dev20240117.img has resolved this issue for me.

[agners]

The original poster reported this on a Raspberry Pi 4, so let this issue be about Raspberry Pi 4 only. On the Raspberry Pi 5 the update system is different so this is a different issue and handled by #3065.

@dyarfaradj what type of USB SSD adapter do you use?

Also for other users affected on Raspberry Pi 4: Are you booting from a USB SSD? If so, which model are you using?

[C0MTRAYA]

Also for other users affected on Raspberry Pi 4: Are you booting from a USB SSD? If so, which model are you using?

got the issue booting from a micro sd

[dyarfaradj]

@dyarfaradj what type of USB SSD adapter do you use?

Also for other users affected on Raspberry Pi 4: Are you booting from a USB SSD? If so, which model are you using?

I'm using this one ORICO 2,5" type C USB 3.1 (Gen 2 6 Gbps) bought from Amazon.

[akm0012]

I'm think I'm seeing the same problem. Here is my output. Going to try to download an older version of HA and see if that helps.

[erkr]

Any chance boot issues like this one on pi4 will be fixed in the upcoming 11.5 release?

[erkr]

@dyarfaradj @akm0012 @C0MTRAYA I'm really interested if 11.5 resolves this issue for you. I'm still on 11.1 waiting till this is resolved (I don't really know myself how to revert an upgrade if this would happen, so I'm waiting). Thanks in advance Eric

[C0MTRAYA]

@erkr ive just read the patch note, issue 3059 dosent seems to be mentionned. Once ill get home ill clone my setup on a dummy sd card and ill try to updare, i will let you know

[erkr]

I hope that the closed issue for PI5 was related: https://github.com/home-assistant/operating-system/issues/3065 and solved this one as well 🤞

[C0MTRAYA]

@erkr unfortunatly, it did not fixed the issue for me.

for what is worth, heres what i get from the console seems the pi is trying to boot from the network since its not seeing the boot partition from the sd card. Took out my test card and put back my main sd card, back online

[dyarfaradj]

So it's not fixed for RB Pi 4 I guess. I can't be bothered to test again since it was a hassle to restore the backup, all settings and re-pair all of my zigbee devices. I'll wait til I see a PR fixing this specific issue

[C0MTRAYA]

@dyarfaradj yeah i have a test card pricesly for testing purpose: if it dont work i can just swap back one of my mains cards and bother to clone the test card later with a working setup for further testing (the number of times I uodated just before going to work only to find something went wrong.. having to wait all day for troubleshooting lol), (that plus two max endurance sd cards cloned weekly, in case of sd corruption..)

[dyarfaradj]

@C0MTRAYA Great idea! I wish I had done this as well. I've always updated to latest HA Core and HA OS and never had any issues til now. So I'll be more careful and take regularly backups. Lets hope this issue gets fixed

[erkr]

@C0MTRAYA I really appreciate this! Have to think how to test these kind of issues on my production system as well. For now I will stick on 11.1 ;-(

[erkr]

I made a full backup and took my chances. I'm glad my upgrade from 11.1 to 11.5 succeeded. My setup uses an ugreen usb3.0 to SSB adapter.

[C0MTRAYA]

so I've deleted the boot partition and formatted my test card, installed a fresh install on 11.5 and it booted (which wasn't on 11.4), restaured from a full backup... and: I've shutdown the host (not just close ha) and unplug the pi for good mesures, plugged it back in and it boot like normal! gonna run it on the test card for a few days and if everything's fine, I'll updates my main cards! @erkr , you inspired me to retry troubleshooting haha! thank you for that!

[erkr]

@C0MTRAYA not much I have done, glad it works for you now! I just upgraded HASOS and no full reboot since the update yet, I now realise that it's a risk as well 😳 Will try that tomorrow 🍀🤞

[C0MTRAYA]

yeah that was the pb on the first place :/ couldn't boot after shutdown; hope the best for you! if it still dont boot you might try what I did

[erkr]

Full reboot worked as well 😅

[foltz61]

On Pi 5. Upgraded from 11.3 to 11.5. Update with a full system reboot went fine. Now I'm gun shy on doing updates quickly. I do regular backups but.....

[sparkydave1981]

Adding a comment here, kinda related... I updated my NUC from HAOS 11.4 to 11.5 last night and also had a hectic boot loop going. Many restarts / power cycling later it's finally stable.

It's a shame we don't get Supervisor logs backed up like HA Core ones from the previous boot.

The only thing I see just before the last restart in my HA Core log is:

2024-02-15 19:59:25.028 ERROR (MainThread) [zigpy.zcl] [0x1FFA:1:0xef00] Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/zigpy/zcl/__init__.py", line 415, in reply
    return await self._endpoint.reply(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/zigpy/endpoint.py", line 278, in reply
    return await self.device.reply(
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/zigpy/device.py", line 488, in reply
    return await self.request(
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/zigpy/device.py", line 322, in request
    await send_request()
  File "/usr/local/lib/python3.12/site-packages/zigpy/application.py", line 833, in request
    await self.send_packet(
  File "/usr/local/lib/python3.12/site-packages/zigpy_deconz/zigbee/application.py", line 527, in send_packet
    status = await req.result
             ^^^^^^^^^^^^^^^^
asyncio.exceptions.CancelledError: Final process shutdown

[github-actions[bot]]

There hasn't been any activity on this issue recently. To keep our backlog manageable we have to clean old issues, as many of them have already been resolved with the latest updates. Please make sure to update to the latest Home Assistant OS version and check if that solves the issue. Let us know if that works for you by adding a comment 👍 This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.